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1.  INTRODUCTION  AND  SUMMARY 


1.1  Objectives 

Because  the  cost  of  airborne  operations  has  increased 
dramatically  in  recent  years,  ground-based  simulators  have  come 
to  play  an  ever-increasing  role  in  the  training  of  Air  Force 
pilots.  Consequently,  one  of  the  major  forces  (if  not  the 
major  force)  driving  training  costs  is  the  number  of  trainee  and 
instructor  hours  required  to  achieve  desired  proficiency  in  the 
training  simulator.  Procedures  that  can  improve  training 
efficiency  have  the  potential  to  improve  the  flying  skills  of 
Air  Force  pilots  while  substantially  reducing  training  lcosts. 

Ground-based  simulators  have  an  advantage  over  airborne 
trainers  in  that  the  informational  (or  perceptual)  environment 
in  which  the  trainee  operates  may,  within  limitations  of  cost, 
be  designed  to  optimize  training.  In  particular,  there  exists 
the  option  to  artificially  enhance  cues  that  are  normally  present 
in  actual  flight  —  and,  perhaps,  to  create  additional  cues  not 
present  in  flight  —  in  order  to  increase  training  efficiency. 

At  present,  however,  there  is  no  detailed,  validated  theory  that 
allows  one  to  predict,  from  knowledge  of  the  informational 
environment,  the  degree  and  rate  of  acquisition  of  flying  skills. 

The  research  summarized  in  this  study  was  directed  toward 
the  long-term  objective  of  developing  an  analytic  tool  for  the 
design  of  training  procedures  and  the  assessment  of  trainee 
performance  in  the  kinds  of  monitoring,  decision,  and  control 
tasks  required  for  flight  management.  A  more  specific  goal  was 
to  extend  the  optimal  control  model  (OCM)  for  pilot/vehicle 
systems  into  a  predictive  tool  that  relates  the  acquisition  of 
continuous  estimation  and  control  strategies  to  the  perceptual 
cueing  environment. 
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Full-scale  development  of  a  model  for  control-strategy 
development  was  beyond  the  scope  of  this  study.  As  a  first  step 
in  this  direction,  the  following  three  major  tasks  were  completed: 

1.  reanalysis  of  existing  manual  control  data  to  determine 

the  effects  of  practice  on  independent  pilot  model  parameters, 

2.  exploration  of  various  hypotheses  concerning  information¬ 
processing  deficits  early  in  suggested  training,  and 

3.  feasibility  testing  of  an  approach  suggested  for  further 
model  development,  in  which  control-strategy  development  is 
related  to  development  of  the  operator's  "internal  model" 
of  the  task  environment. 

In  the  process  of  performing  the  first  task  (model  analysis) , 
we  explored  in  some  detail  the  techniques  adopted  for  performing 
a  statistical  analysis  of  model  parameters.  Results  of  this 
parametric  study  are  contained  largely  in  the  Appendix  for 
readers  interested  in  methodological  detail. 

1 . 2  Approach 

Model  analysis  performed  in  this  study  employed  the  existing 
optimal  control  model  for  pilot/vehicle  systems.  This  model  was 
considered  well-suited  to  the  long-term  modeling  effort  for  a 
number  of  reasons: 

1.  The  optimal  control  model  provides  agreement  with  data  for  a 
wide  variety  of  control  tasks,  including  time-varying  control 
situations . 

2.  The  model  incorporates  many  features  that  derive  from  or  are 
related  to  known  human  performance  characteristics. 

3.  The  concepts  of  internal  models  and  operator  response 
variability,  which  are  integral  to  the  definition  of  control 
strategy  in  the  OCM,  are  extremely  appropriate  in  the  context 
of  learning. 
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4. 


The  model  is  capable  of  dealing  with  cue  utilization  in  a 
direct  fashion,  and  has  been  used  to  model  successfully  the 
influence  of  whole-body  platform  motion  on  asymptotic  tracking 
performance. 

The  following  criteria  were  adopted  to  guide  the  modeling 
effort  pursued  in  this  study,  and  they  are  recommended  for  further 
model  development.  First,  the  model  must  account  for  important 
trends  found  in  the  existing  relevant  data  base.  This  includes 
matching  the  effects  of  various  aspects  of  the  task  environment 
on  both  learning  rates  and  asymptotic  performance  (i.e.,  how 
fast  the  subject  learns,  and  how  well  he  learns). 

Second,  if  one  is  to  develop  a  predictive  model  for  learning 
behavior,  the  model  should  have  a  structure  and  parameterization 
that  facilitate  extrapolation  to  new  situations.  Such  extrapola¬ 
tion  is  enhanced  if  we  can  (1)  determine  a  minimal  set  of 
independent  model  parameters  that  applies  over  the  tasks  of 
interest,  and  (2)  partition  independent  parameters  into  task- 
related  and  operator-related  categories. 

Finally,  model  structure  should  be  compatible  with  what  is 
known  about  human  capabilities  and  limitations.  That  is,  it 
should  "make  sense". 

1.3  Organization  of  this  Report 

The  results  of  this  study  are  presented  at  three  levels 
of  detail.  The  concluding  section  of  this  chapter  provides  an 
extended  summary  of  the  research  effort,  highlighting  key  results. 
Chapters  2  through  5  are  recommended  to  readers  who  wish  to 
understand  the  basis  for  the  results  presented  in  the  Summary. 
Finally,  two  Appendices  provide  further  details  concerning  the 
background  material,  the  results,  and  the  analysis  methodology. 


Chapter  2  --  Background  —  is  devoted  to  three  topics: 

(1)  a  review  of  some  relevant  work  in  skill  acquisition;  (2)  a 
review  of  key  features  of  the  optimal  control  model,  including 
procedures  for  identifying  and  testing  independent  model 
parameters;  and  (3)  a  review  of  some  evidence  that  suggests  a 
relationship  between  the  task  environment  and  the  fidelity  of 
the  operator's  internal  model  of  this  environment. 

Chapters  3-5  summarize,  respectively,  the  results  of  the 
principal  tasks  accomplished  in  this  study:  (1)  analysis  of 
practice  effects  on  pilot  response  behavior,  (2)  hypothesis  testing 
and  (3)  an  exploratory  study  of  an  approach  for  relating  learning 
to  internal  model  development.  Appendix  A  contains  supplemental 
analysis  pertaining  to  some  of  the  background  material  presented 
in  Chapter  2.  Appendix  B  contains  data  for  individual  subjects 
along  with  data  relating  to  methodological  development. 

1.4  Summary 

The  optimal  control  pilot/vehicle  model  used  throughout  this 
study  partitions  the  human  operator's  input/output  behavior  as 
shown  in  the  diagram  of  Figure  1-1.  The  operator  obtains  certain 
task- relevant  perceptual  inputs,  which  are  assumed  to  be  degraded 
by  time  delay  and  "noise"  to  account  for  various  information¬ 
processing  limitations  of  the  human.  The  time  delay  reflects 
neural  conduction  times  and  other  sources  of  pure  transport 
delay.  The  observation  noise  mathematically  accounts  for  various 
sources  of  response  randomness  and  nonlinear  response  behavior 
("pilot  remnant");  for  well-trained  subjects  provided  with  optimal 
control  and  display  environments,  the  observation  noise  identified 
in  simple  laboratory  tracking  tasks  appears  to  reflect  a  limita¬ 
tion  on  the  signal/noise  ratio  with  which  the  operator  can  process 
information. 


NOISY,  DELAYED 
PERCEPTUAL  INPUTS 


STATE 

ESTIMATE 


OPERATOR 

RESPONSE 


Figure  1-1.  Structure  for  Baseline  Model  of  Operator  Response 
Strategy 

The  "optimal  estimator"  and  "optimal  predictor"  elements  of 
the  model  represent  the  operator's  (presumed)  attempts  to  best 
reconstruct  the  current  "state"  of  the  system  he  is  monitoring. 

An  optimal  decision  or  control  law  operates  on  this  state  estimate 
to  provide  the  appropriate  response.  In  the  case  of  a  control 
task,  the  optimal  control  law  includes  a  first-order  lag  — 
characterized  by  the  "motor  time  constant"  —  to  reflect  both 
subjective  and  physiologic  constraints  on  operator  response 
bandwidth. 

Imbedded  in  the  model  structure  of  Figure  1-1  is  an  "internal 
model"  of  the  task  environment  which  accounts  for  all  the  correla¬ 
tions  among  system  variables.  This  internal  model  is  generally 
configured  to  reflect  faithfully  the  statistics  of  the  task 
environment,  but  this  assumption  must  be  challenged  when  we 
consider  tasks  in  which  vehicle  dynamics  contain  significant 
lags,  and/or  subjects  are  incompletely  trained. 


Figure  1-2  shows  that  a  fixed  set  of  independent,  or  "pilot- 
related"  model  parameters  can  yield  a  good  match  to  operator 
frequency  response  (pilot  gain,  phase  shift,  and  "remnant" 
spectrum)  for  a  significant  range  of  laboratory  tracking  tasks. 

If  this  range  is  extended  by  increasing  system  lags,  however,  we 
begin  to  see  a  degradation  in  certain  "pilot  parameters"  as 
■hown  in  Figure  1-3.  Detailed  analysis  of  these  results  suggests 
that  changes  in  the  independent  model  parameters  reflect 
limitations  on  the  operator's  information-processing  capabilities 
that  have  not  been  heretofore  explicitly  identified  —  such  as 
imperfections  in  the  operator's  internal  model  of  the  task 
environment.  Thus,  the  desired  separation  of  operator-  and 
task-related  model  parameters  has  not  been  fully  realized. 

Future  OCM  development  undertaken  to  account  for  learning  behavior 
should  address  this  issue. 

The  effects  of  practice  on  average  operator  frequency  response 
are  shown  in  Figure  1-4  for  three  different  subject  populations, 
operating  in  different  task  environments.  Similar  practice  trends 
were  found:  performance  early  in  practice,  relative  to  near- 
asymptotic  performance,  was  characterized  by  lower  pilot  gain, 
minimal  differences  in  phase  shift,  and  higher  remnant.  Model 
analysis,  the  results  of  which  are  summarized  in  Figure  1-5, 
showed  that  both  the  average  noise/signal  ratio  and  the  motor 
time  constant  decreased  over  the  course  of  the  training  phase. 
Practice-related  changes  were  relatively  greater  and  statistically 
more  significant  for  the  noise  parameter.  These  model  results 
were  obtained  without  constraints  on  the  allowable  changes  in 
model  parameters. 
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Figure  1-5.  Effects  of  Practice  on  Independent  Model  Parameters 

This  data  base  was  subsequently  reanalyzed,  this  time  with 
constraints  imposed  to  reflect  certain  hypotheses  concerning  the 
process  of  skill  acquisition.  The  goal  here  was  to  characterize 
practice  effects  in  a  manner  that  was  parsimonious  and  yet 
applicable  to  a  predictive  model  for  learning. 

We  tested,  individually,  the  hypotheses  that  practice 
effects  could  be  accounted  for  (1)  solely  by  a  reduction  in 
observation  noise/signal  ratio,  (2)  solely  by  an  improving 
ability  to  utilize  rate  cues,  (3)  solely  by  an  improving  internal 
model  of  the  tracking  input,  and  (4)  by  a  combined  reduction  in 
noise/signal  ratio  and  improved  internal  tracking-input  model. 
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Figure  1-6  shows  that  we  can  reject  simple  hypotheses 
concerning  use  of  rate  information  and  internal  model  changes, 
at  least  as  they  apply  to  the  data  base  explored  in  this  study. 

The  observation  noise  hypotheses  provide  the  best  simple 
explanation  for  practice  effects,  and  a  somewhat  closer  match  is 
obtained  if  we  consider  the  combined  effects  of  changes  in  observa¬ 
tion  niose  and  internal  model. 

Because  of  the  substantial  mathematical  development  that 
would  have  been  required,  it  was  beyond  the  scope  of  this  study 
to  explore  hypotheses  concerning  practice-related  changes  in  the 
operator's  internal  model  of  the  simulated  vehicle  dynamics,  at 
least  within  the  framework  of  the  OCM.  Instead,  a  simplified 
analysis  was  conducted  to  demonstrate  that,  were  such  a  model 
development  to  be  undertaken,  it  could  be  expected  to  yield 
reasonable  results. 

A  frequency-response  analysis  was  performed  to  explore  the 
expected  relationship  between  the  operator's  internal  plant  model 
and  certain  aspects  of  the  task  environment:  specifically,  (1)  the 
availability  of  rate  and/or  acceleration  cues,  and  (2)  the 
complexity  of  the  plant  dynamics.  The  qualitative  results  obtained 
in  this  analysis  supported  the  following  explanations  for  some  of 
the  data  trends  shown  in  this  and  previous  studies: 

1.  The  lower  observation  noise  and  motor  time  constant  parameters 
found  for  whole-body  motion  cueing  (Figure  1-5)  are  due  to  the 
more  accurate  internal  plant  model  attainable  in  this  cueing 
environment. 

2.  Learning  is  more  rapid  in  a  motion-base  environment  because 
the  operator  is  more  readily  able  to  determine  the  correct 
order  of  the  plant  dynamics. 

3.  The  apparent  degradation  in  information-processing  capabilities 
associated  with  high-order  plants  (Figure  1-3)  is,  in  part,  a 
reflection  of  a  degraded  internal  model  of  the  task  environment. 
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a)  Observation  Noise 


b)  Lack  of  Rate  Information 


The  results  of  this  suggest  that  we  consider  modifying  the 
OCM  structure  as  shown  in  Figure  1-7.  The  structure  is  similar 
to  that  of  the  baseline  model  shown  in  Figure  1-1,  with  the 
addition  of  a  fourth  adaptive  element  —  the  "optimal  identifier" 
As  the  name  implies,  this  model  element  would  mimic  the  way  in 
which  the  human  operator  identifies  plant  and  input  dynamics,  and 
it  would  use  this  information  to  properly  configure  the  remaining 
adaptive  model  elements.  A  considerable  body  of  literature  in 
the  areas  of  identification  theory  and  adaptive  control  theory 
could  be  drawn  upon  to  aid  such  an  undertaking.  If  properly 
formulated,  the  restructured  OCM  should  be  able  to  address  issues 
not  only  of  operator  skill  development,  but  also  of  performance 
difficulties  (such  as  PIO's)  that  arise  when  substantial  lags 
and  delays  are  introduced  into  the  system. 


Figure  1-7.  Proposed  Structurr  for  Model  of  Operator  Response 
Strategy 
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2 .  BACKGROUND 


Considerable  effort  has  been  devoted  to  understanding  human 
learning  phenomena,  and  to  developing  models  for  human  performance 
in  continuous  control  tasks,  but  very  little  work  has  been  done  in 
developing  quantitative  models  for  how  the  operator  learns 
the  appropriate  control  and  estimation  strategies.  Furthermore, 
what  work  has  been  done  does  not  lend  itself  readily  to  complex 
multi-input,  multi-output  tasks,  nor  does  it  extend  to  the  kinds 
of  monitoring  and  decision  tasks  associated  with  aircraft  flight 
management. 

This  chapter  provides  the  following  background:  (1)  a  review 
of  some  important  concepts  related  to  skill  development;  (2)  a 
brief  description  of  the  optimal-control  pilot  model  including 
techniques  for  identifying,  from  experimental  data,  model  parameters 
associated  with  human  information-processing  limitations;  and 
(3)  a  review  of  relevant  experimental  and  analytical  results,  in 
which  we  demonstrate  what  appears  to  be  an  interaction  between 
the  parameters  of  the  control  task  and  the  quality  of  the  operator's 
internal  model  of  the  task  environment. 

2.1  Skill  Acquisition 

Before  discussing  general  theories  of  human  operator  skill 
acquisition  we  must  first  consider  some  distinctions  that  enter 
into  the  assessment  of  skilled  performance.  Specifically,  we 
consider  the  differences  between  (a)  adaptation  and  learning, 

(b)  performance  and  learning,  and  (c)  level  of  skill  attainable 
and  rate  of  skill  acquisition. 
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Adaptation  vs.  Learning 

If  an  automobile  operator  owns  both  a  stick-shift  and  an 
automatic-shift  vehicle,  he  is  equally  capable  of  operating  either 
one  and  we  say  the  operator  adapts  his  behavior  to  the  vehicle 
currently  being  operated.  However,  if  the  operator  owns  only  an 
automatic  shift  vehicle  and  rents  a  manual  shift  vehicle,  a 
period  of  learning  is  required  to  modify  the  pre-existing  behaviors 
to  meet  the  requirements  of  shift  and  clutch  operation.  Following 


this  period  of  learning,  the  operator  can  call  up  the  relevant 
behaviors  on  demand.  Thus  we  define  adaptation  in  this  context 
as  the  calling  up  of  previously-learned  behaviors  suitable  to  the 
circumstances  and  context  of  concern.  Learning,  on  the  other 
hand,  is  the  acquisition  of  these  desirable  behaviors  on  the  basis 
of  practice  or  repeated  experience. 

Learning,  rather  than  adaptation,  is  the  focus  of  the  work 
described  in  this  report. 


Performance  vs.  Learning  Effects 

Suppose  we  wish  to  evaluate  the  usefulness  for  learning  of 
a  new  display  or  of  the  introduction  of  motion  in  the  training 
simulator.  We  test  two  groups  of  pilots  —  one  with  the  old  display 
(no  motion) ,  and  one  with  the  new  (with  motion) .  While  we  may 
observe  improved  performance  in  the  test  condition,  we  cannot 
conclude  that  greater  learning  has  taken  place  without  examining 
performance  in  a  transfer  condition  that  is  representative  of  the 
conditions  in  which  the  learned  skills  must  be  used.  The  test 
condition  may  produce  a  performance  effect  but  no  learning. 
Similarly,  a  particular  training  condition  may  actually  produce 
worse  performance  but  contribute  importantly  to  more  effective 
performance  upon  transfer  to  the  desired  end  conditions.  Thus, 
only  a  carefully  designed  transfer  experiment  will  distinguish 
between  performance  and  learning  effects. 
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Attainable  Skill  vs.  Rate  of  Skill  Acquisition 

The  effectiveness  of  training  systems  (with  regard  to  skill 
development)  can  be  evaluated  in  terms  of  (1)  the  asymptotic 
level  of  skill  that  can  be  acquired,  given  sufficient  exposure 
to  the  task,  and  (2)  the  rate  at  which  skill  is  acquired.  Although 
one  might  expect  that  a  training  environment  which  maximizes  the 
quality  of  overall  system  performance  also  maximizes  learning 
rates,  one  cannot  be  guaranteed  a  positive  correlation  between 
learning  rate  and  asymptotic  skill  level. 

In  summary,  the  interaction  between  informational  cues, 
asymptotic  skill  level,  and  learning  rate  is  an  issue  of  practical 
importance,  and  is  one  that  must  be  addressed  by  models  that  are 
developed  to  predict  learning  behavior. 

2.1.2  Some  Theories  of  Learning 

The  psychological  literature  is  rife  with  theories  of  learning 
many  of  which  purport  to  apply  to  perceptual-motor  skill  learning. 
Few  of  these  theories  provide  a  suitable  foundation  for 
quantitative  models  for  tasks  related  to  flight  management. 
Habit-strength  theories  fail  to  address  what  is  actually 
learned,  dealinq  only  with  the  successive  build  up 
of  performance  crualitv  (Halqart  and  Bower.  1966)  . 

Skinnerian  or  shaping  theories  suggest  that  improvement  takes  place 
in  small,  incremental  units  and  depend  on  temporally  associated 
rewards,  but  this  can  hardly  be  considered  a  descriptive  theory. 
Stimulus-sampling  or  cue-selection  theories  in  the  S-R  tradition 
have  been  developed  to  a  quantitative  level,  and  at  least  for 
verbal  learning  they  attempt  to  predict  more  than  just  the  shape 
of  a  learning  curve;  but  in  the  skill  context  they  are  not  specific 
with  respect  to  the  characteristics  of  skill  that  are  acquired 
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(Restle  and  Greeno,  1970).  Crossman  (1959)  came  the  closest  when 
he  suggested  that  the  operator  samples  a  set  of  "methods"  for 
accomplishing  a  task  and  on  the  basis  of  some  unspecified 
evaluation  criterion  —  such  as  efficiency  or  effort  —  selects 
out  those  methods  or  strategies  that  are  more  successful  and 
efficient.  However,  Crossman  was  attempting  only  to  describe 
the  characteristics  of  the  learning  curve  and  not  specific 
characteristics  of  skill  acquisition. 

Some  attempts  have  been  made  at  developing  empirical  models 
for  control-strategy  learning  in  terms  of  practice-related 
changes  in  performance  metrics.  Pew  and  Rupp  (1971)  attempted 
to  measure  stages  of  progression  in  terms  of  the  cross-over  model. 
They  found  systematic  differences  in  the  changes  in  these 
parameters  among  children  of  different  ages  and,  in  an  unpublished 
study.  Pew  and  Thomas  mapped  the  coordinate  changes  in  gain  and 
time  delay  as  a  function  of  practice  and  then  introduced  a  time¬ 
sharing  task  to  see  whether  decrements  in  performance  could  be 
described  reasonably  as  regressions  along  the  same  path  in  the 
gain-time  delay  space.  Although  there  were  rather  large 
individual  differences  in  the  acquisition  map  for  different 
subjects,  it  seemed  clear  that  performance  decrements  associated 
with  time-sharing  were  not  simply  regressions  along  the  same  path. 

Smiley,  Reid,  and  Fraser  (1978)  measured  operator  describing 
functions  of  novice  automobile  drivers  during  various  stages  of 
practice  on  an  instrumented  car.  They  found  practice-related 
changes  in  both  amplitude  ratio  and  phase  shift,  which  they 
interpreted  as  changes  in  the  way  the  drivers  were  using  the 
lateral  position  cue  for  steering  control. 

A  few  theoretical  models,  largely  qualitative,  have  been 
proposed  for  control-strategy  development.  Kelley  (1968)  has 
formulated  an  extensive  theory  of  performance  and  skill  acquisition 
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that  builds  on  the  basic  concept  that  learning  consists  of  the 
acquisition  and  refinement  of  an  internal  model  that  defines 
control  strategy  and  relates  it  to  his  model  of  the  environment 
in  which  the  operator  is  controlling.  This  environment  includes 
the  characteristics  of  the  controlled  element  as  well  as  the 
initial  conditions. 

More  recently,  theories  of  skilled  performance  have  intro¬ 
duced  the  concept  of  a  schema  (Pew,  1974;  Schmidt,  1975  and  1976), 
a  generalized  standard  of  performance  from  which  may  be  extracted 
a  particular  instance  for  execution  as  a  motor  program  for  any 
given  set  of  initial  conditions.  The  idea  of  a  schema  is  largely 
compatible  with  the  notion  of  a  general  form  of  internal  model. 

Expanding  on  the  notions  of  schemata,  Rumelhart  and  Norman 
(1976)  propose  three  modes  of  learning:  (1)  the  acquisition  of 
new  data,  using  existing  schemata  to  organize  the  new  information; 
(2)  "tuning"  the  parameters  of  existing  schemata  to  better  fit 
the  data;  and  (3)  restructuring,  whereby  new  schemata  are  developed 
when  existing  memory  structures  are  inadequate  to  account  for  new 
knowledge.  No  mechanisms  are  proposed  for  effecting  these 
changes,  however. 

Another  conceptual  framework  for  control-strategy  development 
which  incorporates  some  of  the  notions  expressed  above  —  is  the 
"Sensory  Organization  of  Perception"  proposed  by  McRuer  and 
colleagues  (Krendel  and  McRuer,  1960;  McRuer  and  Jex,  1967). 
According  to  this  theory,  control-strategy  development  is  assumed 
to  undergo  three  stages:  (1)  development  of  "compensatory" 
skills,  in  which  appropriate  feedback  laws  are  established  for 
stabilization  and  control,  (2)  development  of  "pursuit"  skills, 
in  which  sufficient  knowledge  is  gained  to  allow  application  of 
a  partial  feedforward  strategy,  and  (3)  a  "precognitive  mode"  in 


which  the  pilot  is  able  to  take  full  advantage  of  any  predictability 
inherent  in  the  external  inputs  and  can  act  in  a  generally  "open- 
loop"  fashion. 

A  more  specific  hypothesis  has  been  proposed  by  Fuchs  (1962) 
in  which  the  operator  is  assumed  to  place  increasing  emphasis  on 
velocity  and  acceleration  cues  as  skill  improves.  Although  this 
hypothesis  has  intuitive  appeal,  it  is  not  well  supported  by 
Fuchs'  own  work,  nor  is  it  supported  by  the  results  presented 
later  in  this  report. 

A  scheme  for  both  modeling  and  assessing  operator  control 
behavior,  implemented  by  Greene  et  al  (19  80)  ,  builds  upon  the 
notion  of  internal  modeling  and  attempts  to  account  for  learning 
by  building  up  a  representation  of  plant  dynamics  through 
experience.  "Knowledge"  is  represented  by  a  matrix  of  learned 
relationships  between  tracking  error  and  control  response.  While 
this  model  has  been  able  to  replicate  human  operator  behavior  in 
a  pursuit  tracking  task  using  proportional  dynamics,  it  would 
appear  to  suffer  severely  from  the  "curse  of  dimensionality"  with 
respect  to  modeling  multi-input,  multi-output  high-order  systems. 

2.2  The  Optimal  Control  Model 

The  reader  is  assumed  to  be  generally  familiar  with  the 
optimal-control  model  (OCM)  for  pilot/vehicle  systems.  This  model 
has  been  used  in  numerous  studies  performed  for  AFAMRL  by  BBN 
(Levi son,  Baron,  and  Junker,  1976;  Levison  and  Junker,  1977,  1978; 
Levison  and  Zacharias,  1981).  For  the  reader's  convenience  we 
first  review  the  pilot-centered  components  of  the  model  and  then 
summarize  the  procedure  for  identifying  independent  (i.e.  ,  pilot- 
related)  model  parameters  from  experimental  data. 
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2.2.1  Hfidfil 


We  consider  two  categories  of  pilot-related  model  elements: 
parameters  that  reflect  the  human's  perceptual-motor 
(information-processing)  limitations,  and  elements  related  to  the 
operator's  adaptive  response  strategy. 


The  following  parameters  reflect  perceptual-motor 
limitations: 


Observation  noise.  Each  perceptual  variable  utilized 
by  the  operator  is  assumed  to  be  perturbed  by  a  white 
Gaussian  noise  process  that  is  linearly  uncorrelated 
with  other  pilot-related  or  external  noise  sources.  In 
certain  idealized  laboratory  tracking  situations,  the 
variance  of  the  observation  noise  tends  to  scale  with 
the  variance  of  the  corresponding  display  variable 
(Baron  and  Levison,  1980) ,  in  which  case  we  may 
characterize  this  limitation  by  an  observation 
noise/signal  ratio.  A  more  complex  submodel  for 
observation  noise  may  be  considered  to  account  for 
limitations  such  as  perceptual  thresholds  (Baron  and 
Levison,  1975,1977)  and  attention-sharing  (Levison, 

Elkind  and  Ward,  1971;  Levison,  1979).  In  general,  the 
observation  noise  accounts  for  most  of  the  operator's 
"remnant"  —  the  portion  of  the  control  input  that  is  not 
linearly  correlated  with  external  inputs.  For  trained 
subjects,  remnant  may  reasonably  be  attributed  to 
fundamental  information-processing  limitations  as 
suggested  above  (provided  the  system  to  be  controlled 
is  linear  —  an  underlying  assumption  of  the  OCM) .  For 
untrained  subjects,  observation  noise  may  reflect 
within-trial  variations  in  the  linear  aspect  of  the 
operator's  response  strategy. 


Time  Delay.  A  single  (scalar)  time  delay  is  added  to 
each  display  variable  to  account  for  the  various 
sources  of  delay  associated  with  information 
acquisition,  transformation,  and  response  execution. 

Motor  Time  Constant.  The  operator's  control  response 
is  assumed  to  be  smoothed  by  a  filter  that  accounts  for 
an  operator  bandwidth  constraint.  In  the  model,  this 
constraint  arises  directly  as  a  result  of  a  penalty  on 
control  rate  introduced  into  the  performance  criterion. 
This  constraint  may  mimic  actual  physiological 
constraints  of  the  neuromotor  system,  or  it  may  reflect 
subjective  limitations  imposed  by  the  operator.  The 
time  constant  of  this  first-order  filter  is  called  the 
"motor  time  constant" . 

Motor  Noise.  Just  as  an  observation  noise  is 
postulated  to  account  for  perceptual  and  central 
processing  inadequacies,  a  motor  noise  is  introduced  to 
account  for  an  inability  to  generate  noise-free  control 
actions.  In  many  applications  this  noise  level  is 
insignificant  in  comparison  to  the  observation  noise, 
but  where  very  precise  control  is  important  to  the 
conditions  being  analyzed,  motor  noise  can  assume 
greater  significance  in  the  model.  Early 
implementations  of  the  model  treated  this  noise  as  a 
disturbance  added  to  the  control  response  commanded  by 
the  operator.  In  current  OCM  usage,  motor  noise  is 
generally  added  to  commanded  control  xatfi  in  order  to 
provide  a  better  match  to  low-frequency  response 
behavior  to  the  pilot  describing  function  at  low 
frequencies  (Levison,  Baron,  and  Junker,  1976). 

Cost  Functions.  Except  for  the  cost  weighting  on 
control  rate,  which  we  relate  to  a  motor  time  constant 
as  discussed  above,  the  coefficients  of  the  quadratic 
performance  index  are  generally  considered  as  part  of 
the  task  description,  rather  than  as  human  operator 
limitations.  Nevertheless,  the  operator  can  only 
minimize  what  he  perceives  to  be  the  performance  index. 
To  the  extent  this  perception  differs  from  the  "true" 
performance  index  (as  defined  by  the  experimenter) ,  the 
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performance  index  must  be  considered  as  an  operator- 
related  parameter.  One  might  expect  such  differences 
to  occur  early  in  training. 

The  adaptive  portion  of  the  operator's  response  is 
represented  collectively  by  three  elements  of  the  human  operator 
model:  The  Kalman  estimator,  optimal  predictor,  and  optimal 
control  law.  The  function  of  the  Kalman  estimator  and  predictor 
is  to  generate  the  best  estimate  of  the  current  state  of  system 
variables,  based  on  the  noisy,  delayed  perceptual  information 
available.  It  is  assumed  in  these  elements  that  the  operator  has 
both  an  internal  model  of  the  dynamics  of  the  system  being 
controlled,  and  a  representation  of  the  statistics  of  the 
disturbances  driving  the  system. 

Given  the  best  estimate  of  the  current  system  state,  a  set 
of  control  gains  or  weighting  factors  are  assigned  to  the 
elements  of  the  estimated  state,  in  order  to  produce  control 
actions  that  will  minimize  the  defined  performance  criterion.  As 
might  be  expected,  the  particular  choice  of  the  performance 
criterion  determines  the  weighting  factors,  and  thus  the 
effective  control  law  gains. 


An  automated  gradient  search  scheme  has  been  developed  by 
Lancraft  and  Kleinman  (1979) ,  and  subsequently  modified  by 
Levison  (1981a,  1981b) ,  to  identify  the  pilot-related  model 
parameters  listed  above.  Parameter  values  are  found  that  provide 
a  least-squared  error  joint  match  to  experimental  variance,  gain, 
phase,  and  remnant  measurements.  Identification  is  generally 
performed  under  the  assumption  that  the  test  subject  has  the 
correct  internal  model  of  the  task  environment. 


As  currently  implemented,  the  parameter  identification 
scheme  places  no  constraints  (other  than  non-negativity)  on  the 
identified  values.  Mow,  if  all  independent  model  parameters  are 
allowed  to  vary  freely  to  obtain  a  best  match  to  a  given  data 
set,  all  parameters  will  generally  vary  from  one  data  set  to  the 
next.  In  order  to  interpret  such  results,  we  need  some  method 
for  determining  which  parameter  changes  are  "significant";  that 
is,  which  parameter  changes  are  necessary  to  account  for  changes 
in  operator  response  behavior  due  to  learning  or  to  some  change 
in  experimental  conditions.  Relative  magnitudes  of  various 
parameter  changes  are  not  reliable  indicators  of  significance:  a 
large  change  in  the  value  of  a  particular  model  parameter  may 
simply  reflect  insensitivity  of  the  scalar  modeling  error  to  the 
value  of  that  parameter. 

A  cross-comparison  method  has  been  developed  by  Levison 
(1981a,  1981b)  to  provide  a  qualitative  significance  test  on 
parameter  differences  obtained  from  modeling  the  results  of  two 
experimental  conditions.  This  method  employs  a  numeric,  non- 
analytic  sensitivity  test  as  described  below. 

Assume  that  we  wish  to  analyze  two  data  sets,  corresponding 
to,  say,  the  "baseline"  and  "test"  experimental  conditions; 
specifically,  we  wish  to  determine  whether  or  not  different 
parameter  values  are  required  to  match  these  data.  The  null 
hypothesis,  then,  is  that  a  single  set  of  parameter  values  yields 
a  near-optimal  match  to  the  "baseline"  and  "test"  data. 

To  test  the  null  hypothesis,  we  first  identify  three  sets  of 
pilot  parameters  using  the  gradient  search  scheme:  (1)  the  set 
that  best  matches  the  baseline  data,  (2)  the  set  that  best 
matches  the  test  data,  and  (3)  the  set  that  provides  the  best 


joint  match  to  the  baseline  and  test  data.  For  convenience,  we 
shall  refer  to  the  parameters  identified  in  step  3  as  the 
"average  parameter  set". 

We  next  compute  the  following  four  matching  errors: 

J (B,B)  =  matching  error  obtained  from  baseline  data,  using 

parameters  identified  from  baseline  data  (i.e., 
best  match  to  baseline  data) . 

J (B, A)  =  matching  error  obtained  from  baseline  data,  using 

average  parameter  set. 

J (T,T)  *  best  match  to  test  data. 

J(T, A)  =  matching  error  obtained  from  test  data,  using 

average  parameter  set. 

Finally,  we  compute  the  following  "matching  error  ratios": 
MER(B)  *  J(B,A)/J(B,B) ,  MER(T)  =  J (T,A)/J(T,T)  and,  if  we  wish  to 
reduce  the  results  to  a  single  number,  the  average  of  these  two 
error  ratios. 

In  a  qualitative  sense,  the  greater  the  matching  error 
ratios,  the  more  significant  are  the  differences  between  the 
parameters  identified  for  the  baseline  and  test  conditions.  For 
example,  if  both  matching  error  ratios  are  unity  (the  theoretical 
minimum) ,  then  the  null  hypothesis  is  supported:  there  exists  a 
single  set  of  parameters  that  provides  an  optimal  match  to  both 
data  sets.  Any  differences  between  the  baseline  and  test 
parameter  sets  must  be  considered  insignificant  and  can  be 
attributed  to  imprecision  of  the  identification  procedure. 
Conversely,  if  one  or  both  matching  errors  ratios  are 
substantially  greater  than  unity,  one  must  reject  the  null 
hypothesis  and  consider  the  differences  in  model  parameters  to  be 
"significant";  i.e.,  to  represent  true  differences  in  operator 
response  behavior. 


Interpretation  of  significant  task-related  differences  in 
pilot  parameters  is  contingent  upon  assumptions  concerning  the 
operator's  internal  model  of  the  task.  If  we  can  be  assured  that 
the  internal  model  is  correct,  parameter  differences  may  be 
justifiably  related  to  specific  changes  in  operator  response 
capabilities.  For  example,  both  theoretical  and  experimental 
results  have  led  to  the  association  of  the  observation 
noise/signal  ratio  to  attention-sharing  penalties  among 
concurrent  multiple  tasks  (Levison,  Elkind  and  Ward,  1971; 
Levison,  1979) .  Observation  noise,  as  one  might  expect,  has  also 
been  found  to  reflect  visual  resolution  limitations  in  non-ideal 


display  environments  (Levison,  1971) .  Similarly,  increased  motor 
noise  has  been  found  to  account,  in  part,  for  tracking 
performance  degradation  caused  by  whole-body,  high-frequency 
vibration  (Levison,  1978).  In  these  studies,  performed  with 
subjects  who  were  well-trained  on  wide-band  (i.e.,  low-order) 
control  systems,  the  assumption  of  a  near-perfect  internal  model 
was  reasonable. 

If  the  subject's  internal  model  differs  substantially  from  a 
true  characterization  of  the  task  environment,  parameter  changes 
identified  by  the  above  procedure  may  be  misleading.  For 
example,  suppose  a  test  subject  changes  his  internal  model  from 
one  test  condition  to  the  next,  but  maintains  the  same  inherent 
response  capabilities  in  terms  of  motor  time  constant, 
observation  noise,  etc.  By  modeling  both  sets  of  data  with  the 
assumption  of  an  invariant  (and  correct)  internal  model, 
performance  differences  will,  of  necessity,  be  revealed  as 
changes  in  one  or  more  "pilot"  parameters. 

Now,  we  can  expect  certain  situations  in  which  the  subject 
is  not  likely  to  have  a  near-perfect  internal  model;  e.g.,  tasks 
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requiring  control  of  high-order  systems  with  limited 
informational  cuing,  and  subjects  in  early  stages  of  training. 

The  OCN  has  accordingly  been  modified  to  account  for  various 
imperfections  in  the  operator's  internal  model  (Levison,  Baron, 
and  Junker,  1976;  Baron  and  Berliner,  1977).  A  theory  has  not 
yet  been  developed,  however,  to  predict  the  nature  or  extent  of 
internal  modeling  imperfections. 

2.3  Some  Evidence  for  Imperfect  Internal  Modeling 

Figure  2-1  shows  that  a  fixed  set  of  pilot-related  model 
parameters  can  yield  a  good  qualitative  match  to  frequency  response 
measures  (pilot  gain,  phase,  and  remnant)  for  a  significant  range 
of  laboratory  tracking  dynamics.  (The  match  can  be  improved,  of 
course,  by  "tuning"  the  parameters  for  each  condition.) 

If  we  consider  a  wider  range  of  tasks,  especially  tasks 
employing  dynamics  of  higher  order  and/or  greater  delay,  we  begin 
to  see  a  degradation  in  certain  "pilot  parameters"  with  increasing 
task  difficulty.  As  discussed  later,  these  results  may  reflect 
limitations  on  the  operator's  ability  to  construct  an  accurate 
internal  model  of  the  task  environment. 

Manual  tracking  data  obtained  in  previous  studies  were 
re-analyzed  using  the  parameter  identification  scheme  described 
above.  The  results  of  this  analysis,  some  of  which  have  been 
reported  previously  by  Levison  (1981b) ,  are  summarized  here. 

Four  classes  of  model  parameters  were  identified  from  the 
tracking  data:  observation  noise/signal  ratios,  motor  noise/signal 
ratios,  time  delay,  and  cost  of  control  rate.  Nomenclature  and 
definitions  for  these  parameters  are  as  follows: 
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Figure  2-1.  Fixed-Parameter  Match  to  Three  Tracking  Tasks 


VEHICLE  PHASE  LAG  (dagraw) 

Figure  2-2.  Effects  of  Task  Difficulty  on  Independent  Model 
Parameters 


Pe  =  Observation  noise/signal  ratio  relevant  to  tracking  error, 
in  dB.  This  quantity  is  defined  as  the  ratio  of  the 
observation  noise  covariance  associated  with  perception  of 
tracking  error,  normalized  with  respect  to  tracking  error 
variance.  (Tracking  error  was  a  zero-mean  process.) 

Pe  =  Observation  noise/signal  ratio  relevant  to  perception  of 
tracking  error  rate. 

Pm  =  Pseudo-motor  noise/signal  ratio,  in  dB,  defined  as  the  ratio 
of  pseudo  motor  noise  covariance  normalized  with  respect 
to  the  variance  of  the  commanded  control  rate.  The  prefix 
"pseudo"  signifies  a  noise  process  that  only  influences  the 
pilot's  response  strategy;  it  does  not  reflect  an  actual 
noise  process  injected  into  the  system.  See  Levison,  Baron 
and  Junker  (1976)  for  a  detailed  discussion  of  the  motor 
noise  aspect  of  the  OCM. 

Td  =  Time  delay,  in  seconds. 

G  =  Cost  coefficient  associated  with  control-rate  variance. 

(The  operator  is  assumed  to  adopt  a  response  strategy  that 
minimizes  a  weighted  sum  of  tracking  error  and  control-rate 
variances. ) 

Tm  =  Motor  time  constant,  in  seconds.  This  parameter  is  not 

directly  identified  in  the  search  procedure  but  is  a  derived 
parameter  uniquely  determined  from  the  controlled-element 
dynamics  and  the  identified  G  coefficient.  Because  this 
parameter  is  considerably  less  task-dependent  than  the 
weighting  coefficient  G,  it  is  usually  reported  in  place  of 
G  as  a  "pilot- related"  parameter. 

Parameter  values  for  seven  single-axis  tracking  tasks  are 
shown  in  Table  2-1.  The  plant  dynamics  used  in  these  experiments 
may  be  described  as  follows: 

Configuration  1:  Proportional  control.  (The  pole  at  200 
rad/sec  was  introduced  primarily  to  facilitate  model  analysis.) 
Configuration  2:  Rate  control. 

Configuration  3:  Rate  control  plus  a  2  rad/sec  2nd-order 


Configuration  3: 


Butterworth  filter. 

Configuration  4:  Rate  control  plus  a  1  rad/sec  2nd-order 
Butterworth  filter. 
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TABLE  2-1.  PILOT-RELATED  MODEL  PARAMETERS,  FIXED-BASE  TRACKING 
TRAINED  SUBJECTS 


lonfig. 

Index  I  Plant  Dynamics 


s+200 


K/s 


Pilot-Related  Parameters 


Ref .1  Pe  I  Pe  Td  j  Tm 


2  -22.4 


0.17 

| 

0.082 

0.15 

0.073 

0.26 

0.14 

0.35 

0.18 

0.19 

0.13 

0.21 

0.11 

0.20 

0.13 

=  displacement  observation  noise,  dB 
=  rate  observation  noise,  dB 
Td  =  time  delay,  seconds 
Tm  =  motor  time  constant,  seconds 

2 

G  =  relative  cost  of  control  xjate,  relating  (lbs/sec) 
control  rate  to  (arc-deg)  error. 

♦Observation  noise  of  about  -19dB  associated  with  perception 
error  acceleration. 

♦♦Approximate  pilot  parameters  determined  from  manual  search. 

References:  (1)  Levison  (1981b);  (2)  Levison  (1971);  (3)  Levison 

Lancraft,  and  Junker  (1979);  (4)  Levison  (1980);  (5)  Levison, 
Baron,  and  Junker  (1976) 
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Configuration  5:  Approximate  roll-axis  fighter  response 
characteristics  including  simulator  lag. 

Configuration  6:  Acceleration  control  plus  simulator  lag 
and  delay. 

Configuration  7:  High-order  plant  having  approximate 
acceleration  control  in  the  mid  frequency  range. 

The  K,  K/s ,  and  K/s^  conditions  represented  in  Figure  2-1 
correspond  to  configurations  1,  2,  and  6,  respectively. 

The  principal  model  results  contained  in  Table  2-1  are 
presented  graphically  in  Figure  2-2,  where  we  see  a  trend  toward 
larger  motor  time  constant  and  larger  time  delay  with  increasing 
effective  vehicle  phase  lag  (computed  at  4  rad/sec) .  For  the  most 
part,  these  task- related  differences  in  model  parameters  are 
"significant"  as  defined  above:  they  reflect  differences  in 
operator  response  behavior,  rather  than  imprecisions  in  the 
identification  procedure. 

Now,  since  all  subject  populations  were  well  trained,  and 
since  different  groups  of  subjects  tend  to  perform  similarly  on 
a  given  task  when  well  trained,  it  is  unlikely  that  these 
differences  in  pilot-related  model  parameters  reflect  different 
inherent  information-processing  capabilities  among  the  experimental 
subject  populations.  Rather,  the  following  hypotheses  are  more 
likely  to  explain  the  apparent  trends : 

1.  Subjects  were  motivated  differently  by  the  different  task 
configurations  to  perform  to  capacity. 

2.  Improper  constraints  have  been  imposed  when  applying  the 
OCM  to  this  data  base,  causing  variations  in  what  would 
otherwise  be  relatively  invariant  parameters. 


Differences  in  observation  noise  (Table  2-1)  can  be 
attributed  to  motivational  factors.  For  cases  showing  a  large 
observation  niose  associated  with  error  displacement,  model 
analysis  reveals  that  rms  tracking  error  is  substantially  less 
sensitive  to  noise  on  error  displacement  than  to  noise  on  error 
rate.  Conversely,  the  sensitivities  are  more  nearly  equal  in 
cases  where  similar  observation  noises  are  identified  for  the 
two  perceptual  quantities.  Thus,  the  observation  noise  trends 
are  consistent  with  the  hypothesis  that  the  pilots  are  attempting, 
in  part,  to  minimize  perceptual  workload  by  attending  only  as 
required  to  the  available  perceptual  inputs  (Levison,  1979). 

Time  delay  differences,  however,  cannot  be  attributed  to 
motivational  factors.  For  example,  adding  a  second-order  1  rad/ 
sec  Butterworth  filter  to  K/s  dynamics  caused  the  best-fitting 
time  delay  parameter  to  increase  from  about  0.15  seconds 
(Configuration  2)  to  almost  0.35  seconds  (Configuration  4). 

This  difference  was  found  to  be  significant  using  the  qualitative 
test  described  ab)ve,  and  a  sensitivity  analysis  showed  that 
predicted  tracking  error  variance  was  highly  sensitive  (about  a 
factor  of  2)  to  time-delay  differences  of  this  size.  (See 
Section  A.l,  Appendix  A). 

These  results  indicate  that  (1)  the  subjects  were  capable 
of  performing  with  a  time  delay  parameter  of  under  0.2  seconds 
and  (2),  in  the  case  where  they  exhibited  a  much  larger  delay, 
they  would  have  been  able  to  perceive  the  performance  benefit  of 
reducing  the  delay  to  0.2  seconds  or  less.  Therefore,  we  must 
consider  the  possibility  that  the  larger  delay  reflects  a  limitation 
on  operator  performance  not  directly  reflected  in  the  model 
parameterization  --  a  limitation  not  present  (or,  at  least, 
substantially  less  important)  in  the  wider-band  tracking  tasks. 


One  hypothesis  is  that  the  accuracy  and/or  precision  of  the 
operator's  internal  model  degrades  as  complexity  of  the  controlled 
element  increases,  and  that  this  information-processing  deficit 
is  revealed  (in  the  case  cited  here)  as  a  change  in  time  delay. 

A  number  of  hypotheses  concerning  the  apparent  task-related 
changes  in  the  motor  time  constant  were  explored:  (1)  that  the 
subjects  select  a  motor  time  constant  to  reflect  a  consistent 
tradeoff  between  rms  tracking  error  and  rms  control  activity; 

(2)  that  the  task-varying  time  constant  reflects  a  consistent 
subjective  performance  penalty  on  the  generation  of  control 
activity  rather  than  a  response  bandwidth  limitation,  and  (3) 
that  the  motor  time  constant  reflects  a  response  bandwidth 
limitation  combined  with  a  subjective  penalty  on  control  activity. 

A  sensitivity  analysis  reported  by  Levison  (1981b)  refutes 
the  first  hypothesis.  The  error/control  tradeoffs  identified 
for  the  various  tasks  varied  over  a  wide  range.  Thus,  we  cannot 
relate  differences  in  motor  time  constant  to  differences  in  the 
sensitivity  of  rms  error  to  rms  control  rate. 

Inspection  of  the  right-hand  column  of  Table  2-1  refutes  the 
second  hypothesis.  Relative  cost  of  control  rate  varies  over  two 
orders  of  magnitude  among  the  various  tasks  for  which  meaningful 
comparisons  can  be  drawn.  Clearly,  motor  time  constant  is  a  more 
consistent  descriptor  of  pilot  performance  limitations  than  is 
relative  cost  of  control  rate. 

To  test  the  third  hypothesis  of  a  combined  bandwidth  limita¬ 
tion  and  control-force  penalty,  we  assumed  the  following  submodel 
for  the  cost  coefficient  associated  with  control-rate  variance: 


where  G  is  the  actual  coefficient,  Gq  is  the  component  that 
corresponds  to  some  minimum  motor  time  constant  Tm  and  G  is 
the  component  relating  to  the  subjective  penalty  on  physical 
activity  (i.e.,  control-rate  variances.)  The  assumed  total 
performance  index,  or  cost,  was 

2  7 
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i.e. ,  a  weighted  sum  of  error  and  control-rate  variances. 

Now,  for  the  third  hypothesis  to  be  accepted,  we  need  to 
find  consistent  (i.e.  ,  nearly  invariant)  values  for  Tm  and  G 

o  P 

that  replicate  the  task-related  variation  in  T  shown  in 
c  m 

Table  2-1.  The  analysis  reported  in  Section  A. 1  of  the  Appendix 
to  this  report  yields  mixed  results.  If  we  consider  the  data 
obtained  from  configurations  2,  3,  and  4,  we  can  find  fixed  values 
for  Tmo  and  G^  that  yield  motor  time  constants  not  significantly 
different  from  those  shown  in  the  Table.  On  the  other  hand,  if 
we  consider  the  database  provided  jointly  by  Configurations  2,  4, 
and  6,  fixing  TmQ  and  G^  significantly  degrades  the  model  match, 
relative  to  the  match  that  can  be  obtained  if  these  parameters 
are  allowed  to  vary  across  tasks. 

It  is  possible  that  some  alternative  submodel  structure  would 
satisfactorily  explain  the  variations  in  motor  time  constant 
(e.g.,  a  penalty  on  control  force  instead  of  or  in  addition  to, 
a  penalty  on  rate-of-change  of  control  force) .  An  exhaustive 
search  of  such  model  structures  was  beyond  the  scope  of  this 
effort. 

In  conclusion,  the  analysis  reported  here  is  at  least 
suggestive  of  the  notion  that  task-related  variations  in 
identified  "pilot  parameters"  of  the  OCM  reflect  some  underlying 
limitation  on  human  information-processing  capabilities  that 
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is  not  directly  represented  in  the  OCM  as  heretofore  applied  to 
tracking  tasks.  One  obvious  limitation  to  consider  is  the 
accuracy  and  precision  of  the  operator's  internal  model.  In 
Chapter  5  of  this  report  we  present  a  simplified  analysis  which 
supports  the  intuitive  notion  that  the  operator's  internal 
model  degrades  as  task  complexity  increases. 


3.  EFFECTS  OF  PRACTICE  ON  PILOT  RESPONSE  BEHAVIOR 


In  this  section  we  review  the  effects  of  practice  on  manual 
control  response  behavior  for  three  subject  populations. 
Performance  is  quantified  in  terms  of  standard  control-system 
performance  metrics  as  well  as  "pilot-related"  model  parameters. 

3.1  Review  of  the  Data  Base 

The  data  base  analyzed  in  this  study  was  provided  by  two 
preceding  experimental  studies.  The  study  referred  to  as  the 
"stable  plant"  study  was  conducted  primarily  to  explore  the 
effects  on  pilot  performance  of  a  delay  between  roll-axis  platform 
motion  and  visual  cues  (Levison,  Lancraft  and  Junker,  1971), 
and  employed  simulated  vehicle  dynamics  representative  of  a 
high-performance  figher  in  roll.  The  study  referred  to  as  the 
"unstable  plant"  study  was  conducted  as  part  of  a  research 
project  to  develop  a  tracking  task  for  which  performance  would 
be  highly  sensitive  to  task-  and  environment- related  stress 
(Zacharias  and  Levison,  1979),  and  employed  plant  dynamics  having 
a  divergence  time  constant  of  0.5  seconds.  The  latter  study  was 
conducted  fixed-base. 

All  subject  populations  were  first  given  exposure  to  the 
tracking  dynamics,  without  external  disturbances,  to  allow 
subjects  to  develop  an  appropriate  control  strategy  for  stabiliz¬ 
ing  the  plant  dynamics.  This  initial  exposure  typically  lasted 
a  few  minutes.  A  considerable  period  of  practice  followed, 
spread  out  over  a  number  of  days,  in  which  the  subjects  attempted 
to  minimize  the  effects  of  an  external  forcing  function  on  mean- 
squared  tracking  error.  The  earliest  data  available  for  analysis 
of  the  type  reported  here,  then,  occurred  early  in  the  post¬ 
initialization  phase;  they  do  not  represent  the  subjects'  first 
exposure  to  the  tracking  dynamics. 
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The  stable-plant  tracking  task  consisted  of  maintaining  a 
simulated  figher-like  aircraft  wings  level  in  the  presence  of 
random  turbulence.  Separate  populations  of  4-5  subjects  were 
given  initial  training  on  the  various  conditions  explored  in  this 
study.  "Learning  curves"  are  shown  in  Figure  3-1  for  the  subject 
populations  trained  (a)  fixed  base  and  (b)  moving-base  with 
synchronous  visual  and  motion  cueing.  Averaged  mean-squared 
tracking  error  is  plotted  as  a  function  of  practice  session, 
where  each  session  consisted  of  four  experimental  trials  of 
approximately  three  minutes  each. 

Figure  3-1  shows  that  both  the  fixed-  and  moving-base  subject 
populations  improved  their  tracking  performance  scores  with 
continued  practice,  with  the  latter  subject  group  apparently 
reaching  an  asymptotic  performance  level  during  the  course  of 
training.  The  moving-base  population  not  only  achieved  lower 
error  scores  than  the  fixed-base  population,  but  exponential  fits 
to  the  learning  curves  suggest  that  the  moving-base  population 
reached  asymptote  with  fewer  practice  sessions  (Levison,  Lancraft 
and  Junker) . 

The  unstable-plant  study  required  compensatory  tracking 
using  first-order  unstable  dynamics  having  a  critical 

frequency  of  2  rad/sec.  The  external  disturbance  consisted  of 
a  simulated  first-order  noise  process  injected  in  parallel  with 
the  subject's  control  input. 

Each  subject  successfully  completed  48  training  runs. 

Because  of  initial  difficulty  with  the  task,  subjects  occasionally 
tracked  an  input  disturbance  RMS  level  of  0.5  cm,  rather  than 
the  nominal  1.0  cm.  To  provide  a  fair  base  of  comparison  with 
scores  obtained  under  nominal  conditions,  scores  obtained  with  a 
0.5  cm  RMS  input  were  doubled  under  the  assumption  of  approximate 
operator  linearity.  A  total  of  24  such  run  scores  were  adjusted, 
out  of  a  total  number  of  288  runs  completed  by  the  six-subject 
population. 


Figure  3-2  shows  tracking  score  dependence  on  run  number, 
averaged  across  the  six  subjects.  Population  means  and  standard 
deviations  are  indicated  by  the  dots  and  bars,  respectively. 
Although  48  runs  per  subject  were  conducted,  the  figure  shows 
a  substantial  reduction  in  tracking  score  by  about  the  15th  trial; 
by  the  30th  trial,  tracking  score  was  within  20%  for  the  scores 
obtained  at  the  end  of  training. 

3.2  Effects  of  Practice  on  Operator  Frequency  Response 

We  present  here  a  summary  of  the  effects  of  practice  on 
operator  frequency  response  measures  (gain  and  phase  shift,  and 
remnant).  Data  averaged  across  subjects  are  shown,  along  with 
selected  data  from  individual  subjects.  Data  for  all  subjects 
participating  in  the  stable-plant  and  unstable-plant  studies 
are  presented  individually  in  Appendix  B. 

The  effects  of  practice  on  average  frequency  response  are 
shown  in  Figure  3-3  for  the  three  subject  populations  considered 
in  this  analysis;  (a)  stable  plant,  static  group,  (b)  stable 
plant,  motion  group,  and  (c)  unstable  plant.  Two  stages  of 
practice  are  reflected  here  and  in  the  analysis  to  follow:  very 
early  in  the  training  phase  ("early"),  and  near-asymptotic 
performance  at  the  conclusion  of  the  training  phase  ("late"). 

In  these  figures  (and  in  other  frequency-response  plots  appearing 
in  this  report) ,  0  dB  gain  represents  1  unit  of  control  response 
per  unit  of  tracking  error,  and  0  dB  "remnant"  represents  unit 
control  variance  per  rad/sec  for  the  portion  of  the  operator's 
control  response  not  linearly  correlated  with  the  external 
forcing  function.* 

* 

For  the  stable-plant  study,  error  was  measured  in  degrees  roll 
angle  and  control  was  in  pounds  force;  for  the  unstable-plant 
study,  error  was  in  cm  display  deflection,  and  control  was 
calibrated  in  terms  of  equivalent  cm  of  steady-state  error 
deflection. 


Although  the  sum-of-sines  forcing  functions  contained 
components  ranging  from  about  0.2  to  32  rad/sec,  signal/noise 
limitations  due  to  operator  remnant  limited  the  range  of  valid 
describing  function  measurements.  (See  Levison,  1971,  for  a 
discussion  of  analysis  procedures.)  Thus,  the  range  over  which 
gain  and  phase  measurements  are  plotted  in  Figure  3-3  are  different 
for  the  different  tracking  tasks  and  different  states  of  practice. 

The  three  subject  populations  exhibited  similar  practice 
trends.  Performance  early  in  practice,  relative  to  near-asymptotic 
performance,  was  characterized  by  lower  pilot  gain,  minimal 
differences  in  phase  shift,  and  higher  remnant. 

Five  out  of  the  six  subjects  participating  in  the  unstable- 
plant  study  yielded  frequency  response  measures  very  similar  to 
the  average  response  curves  shown  in  Figure  3- 3c.  Thus,  the 
average  response  may  be  considered  typical  of  an  individual 
subject's  response,  and  model  analysis  of  the  population-average 
response  is  justified  for  that  data  base. 

Intersubject  differences  in  the  practice  trends  were  more 
substantial  for  the  stable-plant  study,  and  these  results  were 
analyzed  on  an  individual  as  well  as  a  group  basis.  Figure  3-4 
compares  frequency-response  measures  obtained  from  two  subjects 
in  the  fixed-base  group  and  two  subjects  in  the  moving-base  group 
of  the  stable-plant  study.  In  both  cases,  the  subject  pairs  were 
selected  to  maximize  intersubject  differences. 

The  remnant  spectrum  showed  the  greatest  variability  with 
respect  to  practice  effects.  Two  subjects  in  Figure  3-4  show  that 
the  remnant  spectrum  obtained  early  in  practice  was  greater  at 
low  frequencies,  and  lower  at  high  frequencies,  than  the  correspond¬ 
ing  remnant  curves  obtained  later  in  the  training  phase.  The 
remaining  two  subjects  showed  larger  low-frequency  remnant  early 
in  training,  but  negligible  practice  effects  on  high-frequency 
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remnant.  As  discussed  below,  model  analysis  suggests  that  these 
different  trends  reflect  differing  practice-related  effects  on 
operator  response  bandwidth. 

3.3  Effects  of  Practice  on  Independent  Model  Parameters 
3.3.1  Analysis  procedures 

The  quasi-Newton  gradient  search  procedure  (Lancraft  and 
Kleinman,  1979;  Levison,  1981a, b)  was  used  to  identify  the 
independent  —  or  "pilot-related"  --  parameters  of  the  OCM  from 


Two  techniques  were  employed  to  determine  the  significance 
of  practice-related  parameter  differences:  (1)  the  qualitative 
cross-comparison  scheme  described  in  Section  2.2.2,  and  (2)  paired- 
difference  t-tests  performed  on  model  parameters.  Application 
of  the  t-test  was  the  same  as  would  be  applied  to  the  reduced 
tracking  data,  except  that  the  identified  parameters  served  as 
the  "data".  As  a  practical  matter,  this  procedure  was  limited 
to  analysis  of  population  means,  with  subject-paired  early/late 
differences  used  in  the  computation  of  the  "t"  statistic.* 


3.3.2  Principle  Results 

Analysis  of  population  trends  is  provided  in  this  Section. 
Parameters  identified  for  individual  subjects  are  presented  in 
Appendix  B. 

Table  3-1  shows  the  effects  of  practice  on  average  pilot 
parameters  for  the  static  (fixed  base)  and  motion  (moving  base) 
subject  populations  participating  in  the  stable-plant  study.  The 
reader  will  notice  invariant  values  for  motor  noise  and  time  delay 
parameters  for  the  "unconstrained"  search  performed  for  the  motion 
population  (lower  half  of  Table  la) .  This  constraint  was  imposed 
after  preliminary  analysis  with  an  unconstrained  procedure  indicated 
that  (1)  practice  had  no  significant  influence  on  these  parameters, 
and  (2)  numerical  convergence  was  substantially  improved  from  this 
particular  data  set  by  imposition  of  these  constraints. 

The  unconstrained  and  constrained  search  procedures  revealed 
similar  trends  for  practice-sensitive  model  parameters.  Both 
observation  noise  and  motor  time  constant  decreased  with  practice 
for  the  two  populations,  with  time  constant  differences  being 
greater  for  the  static  group.  Table  3-la  (static  group)  shows 
negligible  changes  in  motor  noise,  an  increase  with  practice  in 
the  time  delay,  and  apparently  greater  practice  effects  on  position- 
related  noise  as  compared  to  rate-related  noise.  However,  time 


Significance  testing  for  individual  subjects  would  require 
parameter  identification  for  a  number  of  individual  trials  per 
subject,  which  would  generally  entail  significant  computational 
requirements. 
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Table  3-1.  Effects  of  Practice  on  Average  Pilot  Parameters 
Stable-Plant  Study 
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delay  changes  and  differences  between  position  and  rate  noise 
were  found  to  be  largely  insignificant,  thereby  justifying  the 
constrained  search  reflected  in  Table  3-lb.  (The  effects  of 
imposing  these  constraints  on  the  matching  error  are  tabulated 
in  Appendix  B.  ) 

Results  of  the  significance  test  of  practice-related 
parameter  differences  are  given  in  Table  3-2.  The  t-tests 
indicated  that  only  observation  noise  differences  are  significantly 
influenced  by  practice;  this  was  true  for  both  the  static  and 
motion  groups.  The  qualitative  cross-comparison  test  confirmed 
this  conclusion  for  the  motion  group.  For  the  static  group, 
however,  changes  in  motor  time  constant,  but  not  observation 
noise,  were  found  to  be  significant. 

The  discrepant  conclusions  yielded  by  the  two  significance 
testing  methods  may  be  due,  in  part,  to  appreciable  subject-to- 
subject  differences.  Figure  3-5,  which  shows  parameters  identified 
for  individual  subjects,  reveals  an  apparent  negative  correlation 
between  practice- related  effects  on  observation  noise  and  motor 
time  constant  for  the  static  group.  That  is,  subjects  exhibiting 
relatively  large  changes  in  motor  time  constant  tended  to  exhibit 
relatively  small  changes  in  observation  noise,  and  vice  versa. 

This  finding  suggests  some  sort  of  tradeoff  between  decrements  in 
motor  time  constant  and  decrements  in  observation  noise  in  the 
early  stages  of  practice.  Because  the  parameter  differences  shown 
in  Figure  3-5  were  found  to  be  largely  significant  (i.e.,  matching 
error  ratio  greater  than  2) ,  this  presumed  tradeoff  does  not  simply 
reflect  an  insensitivity  of  the  model-matching  scheme.  It  is 
possible  that  performance  was,  to  some  extent,  insensitive  to 
piloting  strategy  early  in  training,  with  a  consequent  increase  in 
inter-subject  variability. 


Table  3-2  .  Significance  Test  of  Practice-Related  Parameter 
Differences,  Stable-Plant  Study 
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Figure  3-5.  Effects  of  Practice  on  Pilot-Related  Model 
Parameters,  Stable-Plant  Study 


Results  of  the  model  analysis  were  more  consistent  for  the 
unstable-plant  study.  Table  3-3  shows  that  practice  influenced 
primarily  the  observation  noise/signal  ratio  for  the  subjects 
participating  in  this  study.  Both  methods  for  determining  signifi¬ 
cance  indicated  that  only  the  noise  parameters  were  significantly 
effected.  On  the  average,  motor  time  constants  and  observation 
noise/signal  ratios  were  lower  than  those  obtained  in  the  motion- 
cue  study,  perhaps  because  of  the  more  severe  information-processing 

requirements  imposed  by  the  unstable  plant  dynamics  used  in  the 
unstaPle  plant  study. 


Table  3-3.  Effects  of  Practice-Related  Model 
Parameters,  Unstable-Plant  Study 

a)  Model  Parameters 
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4.  HYPOTHESIS  TESTING 

The  model  results  presented  in  the  previous  section  were 
obtained  without  imposing  pre-conceived  notions  as  to  what  the 
effects  of  practice  might  be.  All  pilot-related  independent 
model  parameters  were  initially  allowed  to  vary  at  will  across 
conditions;  only  after  determining  that  certain  parameters  were 
insensitive  to  practice  effects  were  constraints  subsequently 
imposed  on  the  search  procedure.  In  this  initial  analysis  phase, 
then,  the  OCM  was  used  strictly  as  a  diagnostic  tool.  As  the 
next  step  toward  developing  a  predictive  model  for  learning,  the 
data  base  was  re-analyzed,  this  time  with  constraints  imposed  to 
reflect  various  hypotheses  concerning  the  process  of  skill 
acquisition.  The  goal  of  this  second  analysis  phase  was  to 
characterize  learning  effects  in  a  manner  that  was  parsimonious 
and  yet  applicable  to  a  predictive  model  for  learning. 

We  explore  here  various  hypotheses  concerning  information¬ 
processing  deficits  early  in  the  training  phase,  relative  to 
information-processing  capabilities  after  substantial  practice. 
Some  of  these  hypotheses  are  suggested  by  the  results  presented 
in  the  preceding  chapter,  others  by  existing  theories  of  learning. 
Potential  deficits  are  first  explored  individually,  then  in 
combination.  All  model/data  comparisons  are  with  respect  to 
frequency-response  data/averaged  across  subjects  within  a  given 
population  (5  subjects  for  the  static  group,  stable-plant  study; 

4  subjects  for  the  motion  group,  stable-plant  study/  and  6 
subjects  for  the  unstable-plant  study) . 

One  should  keep  in  mind  that  the  "early"  practice  data 
reflect  operator  performance  only  after  the  operator  has  learned 
to  stabilize  the  plant  and  is  able  to  track  for  a  three-  to  four- 
minute  period.  Therefore,  performance  early  in  training  does  not 
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necessarily  reflect  operator  capabilities  and  strategies  upon 
first  exposure  to  the  tracking  task.  Although  steady-state 
data  were  generally  obtainable  after  relatively  brief  exposures 
to  the  task,  one  must  consider  the  possibility  that  a  substantial 
amount  of  learning  (especially  regarding  plant  dynamics)  occurred 
between  first  exposure  and  the  "early"  practice  results  reported 
here. 

4.1  Increase  in  the  Observation  Noise/Signal  Ratio 

The  analysis  reported  in  the  previous  chapter  suggests  that 
observation  noise  differences  should  account  for  much  of  the 
practice  effects.  The  following  test  was  therefore  performed: 

(1)  independent  model  parameters  were  initially  set  to  the 
values  identified  by  the  constrained-search  procedure  from  the 
late-practice  data,  and  (2)  the  observation  noise/signal  ratio 
was  increased  until  the  early-practi'ce  average  MSF,  score  was 
matched  to  within  10%.  A  fixed  noise/signal  ratio  was  maintained 
for  the  position  and  rate  components,  and  all  other  parameters 
were  kept  fixed  at  the  values  appropriate  for  late  practice. 

The  objective  of  this  analysis,  then,  was  to  determine  how  well 
practice  effects  could  be  accounted  for  by  a  change  in  a  single 
model  parameter;  specifically,  observation  noise/signal  ratio. 

The  effects  of  changing  the  observation  noise  are  shown  in 
Figure  4-1.  In  order  that  one  may  qualitatively  judge  the  accuracy 
of  the  model  trends,  average  frequency-response  experimental  data 
are  shown  for  comparison. 

As  the  reader  will  shortly  see,  this  hypothesis  provides  the 
best  simple  explanation  for  observed  performance  trends.  For  all 
three  tracking  tasks,  predicted  gain  is  decreased  and  predicted 
remnant  is  increased. *  Not  all  details  of  the  early/late 
_ 

Throughout  this  discussion,  a  comparison  of  early-practice 
behavior  to  late-practice  behavior  is  implied. 


differences  are  modeled,  however.  In  particular,  the  model 
predicts  that  remnant  will  be  increased  at  all  measurement 
frequencies;  the  experimental  data  show  that  high-frequency 
remnant  was  either  unmodified  or  slightly  decreased.  Furthermore, 
Figure  4-lc  shows  a  greater  effect  on  the  high-frequency  gain  peak 
than  was  observed.  (Valid  high-frequency  describing  function 
estimates  were  not  obtained  for  the  stable-plant  tracking  tasks.) 
There  also  seems  to  be  a  tendency  for  predicting  too  much  phase 
lag  at  mid-to-high  frequencies. 

4.2  Lack  of  Information  on  Error  Derivatives 

It  has  been  suggested  that  an  untrained  operator  makes 
relatively  little  use  of  rate  information  provided  by  the  velocity 
of  the  error  indicator,  and  that  the  reliance  on  this  type  of 
information  increases  with  continued  practice  (Fuchs,  1962).  To 
test  the  validity  of  this  hypothesis,  model  parameters  were  initially 
selected  as  in  the  previous  test,  and  the  "display  vector"  was 
reduced  to  a  single  quantity:  error  displacement.  Thus,  rate  cues 
were  eliminated  from  the  analysis  of  the  fixed-base  tasks,  and  both 
rate  and  acceleration  cues  were  omitted  from  the  analysis  of  the 
moving-base  task. 

Figure  4-2  shows  that  this  hypothesis  produces  some  predicted 
performance  trends  that  are  counter  to  experimental  observations. 
Although  the  effects  of  practice  on  low-frequency  gain  are  matched 
reasonably  well,  too  much  high-frequency  phase  lag  is  predicted, 
and  the  predicted  remnant  trends  are  wrong.  Specifically,  this 
hypothesis  predicts  that  remnant  will  be  increased  largely  at  high 
frequencies,  whereas  the  data  show  the  increase  to  be  primarily  at 
low  frequencies.  Thus,  the  hypothesis  of  a  specific  deficiency 
in  extracting  and  utilizing  velocity  information  early  in  practice 
is  not  supported. 


4.3  Deficient  Internal  Model  of  the  Tracking  Input 

As  noted  above,  the  "early"  results  presented  here  do  not 
represent  the  subjects'  first  exposure  to  the  tracking  tasks, 
but  rather  the  first  available  opportunity  to  obtain  consistent 
performance  measures  once  the  subjects  have  learned  some  rudimentary 
strategy  for  stabilizing  and  controlling  the  vehicle.  Consequently, 
it  is  possible  that  a  substantial  portion  of  the  learning  taking 
place  during  the  practice  intervals  relevant  to  these  data  bases 
was  devoted  to  learning  the  statistics  of  the  external  inputs. 

This  hypothesis  is  consistent  with  the  "Sensory  Organization  of 
Perception"  theory  (Krendel  and  McRuer,  1960;  McRuer  and  Jex, 

1967) ,  in  which  control-strategy  development  is  assumed  to  undergo 
three  stages:  (1)  development  of  "compensatory"  skills,  in  which 
appropriate  feedback  laws  are  established  for  stabilization  and 
control;  (2)  development  of  "pursuit"  skills,  in  which  sufficient 
knowledge  is  gained  to  allow  application  of  a  partial  feedforward 
strategy;  and  (3)  a  "precognitive"  mode  in  which  full  advantage 
is  taken  of  any  predictability  of  the  external  inputs. 

In  keeping  with  these  notions,  a  third  hypothesis  was  tested; 
namely,  that  performance  deficiencies  early  in  practice  can  be 
accounted  for  by  a  deficient  internal  model  of  the  input  statistics. 
To  test  this  hypothesis,  the  (mathematical)  "pilot”  was  assumed 
to  have  no  knowledge  of  the  correlations  inherent  in  the  forcing 
function.  A  white-noise  (internal)  model  of  the  input  was 
assumed,  and  the  covariance  of  this  white  noise  was  adjusted  to 
match  early-practice  MSE  scores,  with  other  pilot-related  parameters 
fixed  at  values  appropriate  for  asymptotic  tracking. 

Figure  4-3  shows  that  practice  effects  on  pilot  gain  are 
matched  over  much  of  the  spectrum.  However,  low-frequency  phase 
effects  (not  seen  in  the  data)  are  predicted,  and  the  predicted 
remnant  spectrum  is  uniformly  too  low.  Thus,  the  notion  of  a 


deficient  internal  representation  of  the  tracking  input  is  not 
sufficient  as  a  stand-alone  hypothesis  to  account  for  practice 
effects . 

While  it  is  reasonable  to  expect  that  the  subject's  internal 
model  of  the  plant  was  also  being  refined  during  practice,  model 
analysis  was  restricted  to  consideration  of  input  modeling  for 
two  reasons.  First,  because  of  the  mathematics  involved,  considera 
tion  of  certain  types  of  input-model  deficiencies  can  be  treated 
with  considerably  less  computational  cost  than  plant-model  deficien 
Second,  the  plant  can  be  stabilized  with  a  grossly  deficient  input 
model,  and  one  has  a  rationale  for  choosing  such  a  deficient  model 
(e.g. ,  the  untrained  subject  is  unaware  of  correlations  within  the 
input  signal).  On  the  other  hand,  severely  deficient  plant  models 
cannot  be  explored  because  of  resulting  closed-loop  instabilities, 
and  there  is  no  correspondingly  obvious  candidate  for  a  plant 
model  deficiency. 

4.4  Combination  of  Observation  Noise  and  Deficient  Input  Model 

The  foregoing  results  suggested  a  test  of  the  combined 

hypothesis  of  increased  observation  noise  and  a  deficient  input 
model  early  in  training.  A  gradient  search  was  performed  jointly 
on  the  internal  white-noise  input  covariance,  and  the  observation 
noise/signal  ratio,  to  provide  the  best  overall  match  to  the 
early-practice  data.  Remaining  parameters  were  fixed  at  values 
appropriate  to  asymptotic  performance.  Figure  4-4  shows  that,  on 
balance,  a  good  match  to  the  early  data  was  achieved  with  this 
hypothesis. 

4.5  Summary  of  the  Hypothesis-Testing  Exercise 

The  second  analysis  phase  was  begun  with  essentially  a  two- 
parameter  match  to  practice  effects  (observation  noise/signal  ratio 


PHASE(d«g)  GAIN  (dB) 


and  motor  time  constant) ,  and  it  concluded  with  a  different  two- 

parameter  match  (observation  noise/signal  ratio  and  a  deficient 

* 

internal  model) .  Since  either  treatment  is  able  to  provide  an 
acceptable  data  match,  the  decision  concerning  further  model 
development  must  be  guided  by  the  remaining  criteria  proposed  in 
the  introduction;  namely,  one  should  pursue  an  approach  that 
makes  sense  in  terms  of  psychomotor  capabilities  and  is  most 
likely  to  lead  to  a  model  of  predictive  value. 

Practice-related  reduction  in  observation  noise  makes 
intuitive  sense.  Since  observation  noise  is  the  mathematical 
device  by  which  most  of  the  pilot's  "remnant"  is  accounted  for, 
time  variations  and  nonlinearities  are  reflected  in  the  observation 
noise.  One  would  expect  that  continued  practice  would  bring  about 
a  more  linear  and  stable  response  strategy. 

One  might  also  expect  continued  practice  to  enhance  the 
operator's  overall  motor  capability,  which  might  well  be  reflected 
as  a  practice- related  reduction  in  the  motor  time  constant. 

However,  as  discussed  in  Chapter  2,  extensive  analysis  of  the  data 
base  suggests  that  the  apparent  changes  in  motor  time  constant 
are  more  directly  related  to  the  perceptual  environment  than  to 
direct  "motor  training".  For  example,  the  subject  population 
trained  on  the  moving-base,  stable-plant  task  had  a  lower  average 
motor  time  constant  early  in  practice  than  the  fixed-base  popula¬ 
tion  had  late  in  training.  Since  the  subject  populations  were 
matched  for  tracking  ability  prior  to  the  study,  it  is  not  likely 
that  the  apparent  differences  in  motor  time  constants  reflected 
inherent  differences  in  response  bandwidth  capabilities.  On 
balance,  the  evidence  suggests  that  practice-  and  plant-related 
motor  time  constant  represent  differences  in  "perceptual  efficiency" 
due  perhaps  to  differences  in  the  quality  of  the  operator's 
internal  model. 

★ 

That  is,  variations  in  only  two  parameters  are  required  to  account 
for  early/late  performance  differences.  To  match  performance  in  a 
specific  condition,  non-zero  values  are  typically  associated  with  at 
least  four  " pi lot- related"  parameters,  given  the  current  model  structure 
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If  we  accept  the  notion  that  the  subject's  internal  model 
changes  with  practice,  we  should  be  prepared  to  consider  changes 
in  the  internal  model  of  the  plant  as  well  as  of  the  input . 
Furthermore,  subsequent  model  development  should  address  the 
problem  of  predicting  how  the  internal  model  (and  the  observation 
noise/signal  ratio)  changes  with  practice,  and  how  these  practice 
effects  are  modified  by  the  details  of  the  task  environment. 

These  issues  are  addressed  in  a  preliminary  manner  in  the  follow¬ 
ing  chapter. 


5. 


INTERNAL  MODEL  DEVELOPMENT:  A  PRELIMINARY  TREATMENT 


The  practice  effects  explored  in  the  preceding  two  chapters, 
along  with  some  earlier  results  with  high-order  control  systems, 
suggest  that  development  of  the  operator's  internal  model  of  the 
conti’l  system  is  an  important  aspect  of  the  "learning"  that  takes 
place  in  control  (and  presumably  monitoring  and  decision)  tasks. 

One  way  of  treating  internal  model  development  is  to  add 
a  fourth  adaptive  element  —  the  "optimal  identifier"  —  to  the 
OCM  model  structure  as  indict.  led  in  Figure  5-1.  As  the  name 
implies,  this  model  element  would  mimic  the  way  in  which  the 
human  operator  identifies  plant  and  input  dynamics,  and  it  would 
use  this  information  to  properly  configure  the  remaining  adaptive 
model  elements. 

Because  of  limited  resources,  a  highly  simplified  model 
analysis  was  performed:  one  not  in  keeping  with  the  framework 
of  the  OCM,  but  yet  sufficient  to  allow  a  feasibility  test.  A 
two-operator  task  was  considered,  in  which  a  "pilot"  performed 
the  roll-axis  tracking  task  used  in  the  stable-plant  study 
referenced  above,  and  an  independent  "observer"  attempted  to 
identify  the  plant  dynamics.  The  observer  performed  this  identifi¬ 
cation  on  the  basis  of  the  pilot's  control  -input  and  the  tracking 
error,  and  was  prevented  from  making  a  perfect  identification 
because  of  the  effects  of  the  tracking  input  (not  directly  available 
to  the  observer) .  The  observer  performed  the  identification  by 
adjusting  the  parameters  of  a  fixed-form  model  to  reduce  the 
mean-squared  difference  between  "predicted"  and  observed  tracking 
error . 

A  signal  flow  diagram  of  the  hypothesized  measurement  situation 
is  shown  in  Figure  5-2.  All  signals  and  system  elements  in  this 
diagram  are  Laplace- transformed  variables. 
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Figure  5-2.  Flow  Diagram  of  Hypothesized  Identification  Task 
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The  closed- loop  tracking  task  is  represented  by  the  upper 
dotted  box.  The  human  operator  ("pilot”)  H  responds  to  tracking 
error  E  by  generating  a  control  input  U.  This  control  input  is 
corrupted  by  an  uncorrelated  noise  component,  or  remnant,  R. 

The  control  signal  is  added  with  the  disturbance  input  I  to  form 
the  total  input  C  to  the  controlled-element  ("vehicle")  V. 

The  observer  (represented  by  the  lower  dotted  box)  is  assumed 
to  have  perfect  sensing  of  the  tracking  error  and  the  operator’s 
total  control  input  (i.e.,  the  sum  of  the  input-correlated  and 
remnant- re la ted  control  actions) .  The  observer  is  assumed  to 

A 

generate  a  model,  or  estimate,  of  the  vehicle  V,  which  yields 

A 

a  predicted  tracking  error  E  when  excited  by  the  pilot's  control 
input.  The  observer  adjusts  the  estimated  vehicle  dynamics  to 
yield  minimum  mean-squared  estimation  error  Z. 

The  following  spectral  quantities  are  computed  from  Figure 

5-2  : 


e  •  e .  = 
1  i 


(5-1) 


1  +  HV 


urur  = 


(5-2) 
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(5-3) 


where  subscripts  "i"  and  "r"  represent  input-correlated  and 
remnant -re la ted  signal  components.  Making  use  of  the  relation¬ 
ships  expressed  in  Equations  5-1  and  5-2,  we  write  Equation  5-3 


The  vehicle  dynamics  "identified"  by  the  observer  is  the  V  that 
minimizes  mean-squared  Z,  and  the  spectrum  of  this  minimal  Z 
is  taken  as  the  estimate  of  the  tracking  input.  That  is. 


$  _  <J> 

ii  ZZ 


(5-5) 


The  above  analysis  is  predicated  on  the  assumption  that  the 
observer  adjusts  his  model  to  minimize  the  difference  between 
the  predicted  and  actual  tracking  error.  Alternatively,  the  model 
may  be  adjusted  to  minimize  the  prediction  error  for  any  derivative 
(or  combination  of  derivatives)  of  the  tracking  error.  In  the  case 

/V 

of  the  nth  derivative,  the  model  V  is  adjusted  to  minimize  the 
power  contained  in 


Note  that  the  identification  task  posed  here  is  the  mathe¬ 
matical  dual  of  the  problem  of  identifying  pilot  response  behavior. 
In  this  case,  identification  is  enhanced  by  the  occurrence  of 
large  remnant  power  and  small  tracking  input  power.  For  zero 
tracking  input,  $ zz  is  minimized  by  setting  V=V;  that  is, 
perfect  identification  results.  On  the  other  hand,  if  remnant 

A 

is  zero,  $zz  is  minimized  by  setting  V  =  -1/H.  The  prediction 
that  pilot  remnant  will  aid  the  identification  process  is 
consistent  with  notions  of  system  identification  that  have 
appeared  in  the  control  literature. 

No  experimental  data  were  used  for  this  analysis.  Instead, 
tracking  "data"  were  generated  by  the  OCM  using  nominal  values 
for  pilot- related  parameters.  Analysis  was  performed  in  the 
frequency  domain.  Parameters  of  the  analysis  included  plant 
dynamics,  perceptual  cues  available  to  the  observer,  and  the 
order  of  the  internal  model  adopted  by  the  observer. 


Because  of  the  preliminary  nature  of  this  modeling  effort, 
the  following  results  are  not  intended  as  accurate  predictions 
of  a  human  operator's  ability  to  identify  system  dynamics,  but 
rather  to  explore  gross  trends  relating  internal  models  to 
elements  of  the  task  environment. 

The  results  of  four  tests  performed  with  this  approximate 
model  are  shown  in  Figure  5-3.  We  first  describe  the  test  results 
and  then  discuss  the  implications  with  regard  to  pilot  model 
development. 

To  explore  the  influence  of  the  cueing  environment  on  the 
operator's  internal  model  of  the  plant,  analysis  was  performed 

with  error  displacement,  rate,  or  acceleration  information  assumed 

* 

available  to  the  operator.  The  observer  was  assumed  to  have  a 
full-  (in  this  case,  second-)  order  model  of  the  plant.  Figure 
5-3a  shows  that  the  transfer  function  of  the  identified  vehicle 
dynamics  (continuous  curves)  progressively  approached  the  true 
plant  response  (discrete  points)  as  higher-order  information  was 
made  available. 

A  subsequent  analysis  explored  the  interaction  between  the 
available  perceptual  cues,  the  order  of  the  observer's  internal 
model,  and  the  identified  plant  dynamics.  Error-rate  and  error- 
acceleration  cues  were  considered,  as  were  first-  and  second- 
order  plant  models.  Figure  5-3b  shows  that,  with  only  error- 
rate  available,  the  order  of  the  internal  model  had  little 
influence  on  how  accurately  the  plant  transfer  function  was 
identified  (except  at  the  highest  frequencies,  where  signal  power 
is  relatively  low).  However,  when  acceleration  cues  were 
available  (Figure  5- 3c),  the  second-order  plant  model  gave  a 
noticeably  closer  match  to  the  true  plant  dynamics. 

*For  simplicity,  combinations  of  these  cues  were  not  considered, 
although  in  practice  one  would  expect  a  human  operator  to  use  all 
relevant  information  available. 


A  final  test  was  performed  to  explore  the  effects  of  plant 
dynamics  on  the  fidelity  of  the  predicted  internal  plant  model. 

Two  sets  of  plant  dynamics  were  considered:  simple  rate  control 
(designated  as  "unfiltered"  dynamics  in  Figure  5-3d) ,  and  rate 
control  cascaded  with  a  second-order  Butterworth  filter  having 
a  break  frequency  of  1  rad/sec  ("filtered"  dynamics).  In  each 
case,  the  observer  was  assumed  to  have  an  internal  model  of  the 
appropriate  order.  Figure  5-3d  shows  that  a  qualitatively  better 
estimate  of  plant  dynamics  was  obtained  for  the  lower-order  plant. 

The  trends  revealed  by  this  analytical  feasibility  study  are 
consistent  with  the  experimental  trends  reported  above:  conditions 
which  led  to  an  improvement  in  the  predicted  internal  model  of 
the  plant  correspond  to  experimental  conditions  that  led  to  an 
apparent  increase  in  information-processing  efficiency.  Further¬ 
more,  certain  cause-and-ef feet  relationships  are  suggested: 

1.  The  results  of  Figure  5-3a  suggest  that  the  reduced  values  for 
pilot- related  model  parameters  associated  with  whole-body 
motion  cueing  are  due  to  improved  internal  plant  model 
attainable  in  this  cueing  environment. 

2.  Figures  5- 3b  and  5- 3c  suggest  that  learning  is  more  rapid 
in  a  motion-base  environment  partly  because  the  operator  is 
more  readily  able  to  determine  the  correct  order  of  the 
plant  dynamics. 

3.  Figure  5- 3d  supports  the  notion  that  the  apparent  degradation 
in  information-processing  capabilities  associated  with  high- 
order  plants  is,  in  part,  a  reflection  of  a  degraded  internal 
model  of  the  task  environment. 

It  must  be  re-emphasized  that  the  model  results  summarized 
in  Figure  5-3  are  highly  qualitative  and  are  intended  only  to 
illustrate  trends.  The  intent  of  this  final  analysis  phase  has 
been  to  demonstrate  the  feasibility  of  considering  deficient 
internal  models  and  of  pursuing  mechanisms  to  predict  such 
deficiencies.  The  preliminary  results  reported  here  suggest  that 
such  an  approach  is  feasible. 


APPENDIX  A 


SUPPLEMENTAL  ANALYSIS  PERTAINING  TO  BACKGROUND  MATERIAL 

This  appendix  summarizes  additional  model  analysis  relevant 
to  the  material  presented  in  the  background  section  (Chapter  2). 
Specifically,  differences  in  time  delay  and  in  motor  time  constant 
across  experimental  conditions  are  explored. 

A. 1  Importance  of  Time  Delay  Differences 

Sensitivity  analysis  was  performed  to  explore  the  importance 
of  the  differences  in  the  time  delays  identified  for  a  simple 
rate-control  task  (Configuration  2,  Table  2-1),  and  for  a  task 
using  rate-control  cascaded  with  a  low-bandwidth  Butterworth 
filter  (Configuration  4,  Table  2-1).  Time  delays  were  0.15  and 
0.35  seconds,  respectively,  for  these  two  cases. 

Two  types  of  sensitivity  analyses  were  performed.  First,  the 
qualitative  significance  test  described  in  Section  2.2.2  was 
performed  to  determine  whether  or  not  the  observed  time  delay 
difference  was  significant  in  terms  of  model-matching  error. 

Matching  errors  of  14.0  and  7.92  were  obtained  for  the  rate-  and 
filtered-rate-control  cases,  respectively,  with  all  independent 
parameters  adjusted  for  minimum  matching  error.  A  second  set  of 
matching  errors  —  25.9  and  20.8,  respectively  --  were  obtained 
for  these  two  cases  with  time  delay  fixed  at  the  average  value  of 
0.25  seconds.  Matching  error  ratios  of  18.5  and  2.6  were  computed 
from  these  results,  indicating  that  the  task- related  change  in 
time  delay  was  "significant"  according  to  the  criterion  error 
ratio  of  2.0. 

The  four  remaining  independent  model  parameters  (two  observa¬ 
tion  noises,  motor  noise,  and  motor  time  constant)  were  reoptimized 
during  the  fixed-delay  matching  exercises.  Thus,  the  task- 
related  time-delay  change  cannot  be  compensated  for  by  readjustments 
of  one  or  more  remaining  model  parameters. 


To  explore  the  potential  importance  of  motivational  factors, 
a  second  sensitivity  test  was  performed  to  determine  the  effect  of 
time  delay  on  tracking  error  performance  for  the  filtered- rate 
plant.  Error  scores  were  predicted  for  time  delays  of  0.15  and 
0.35  seconds,  with  remaining  model  parameters  fixed  at  the  values 
that  provided  the  best  match  to  the  experimental  data.  The  larger 
delay  yielded  a  predicted  rms  tracking  error  score  nearly  60% 
greater  than  that  predicted  for  the  smaller  delay.  Thus,  given  the 
particular  model  structure  applied  to  this  analysis,  we  fail  to 
support  the  hypothesis  that  the  larger  time  delay  is  due  to 
indifference  on  the  part  of  the  subjects. 

A. 2  Analysis  of  Motor  Time  Constant  Differences 

Some  of  the  previous  results  represented  in  Table  2-1  were 
reanalyzed  to  test  the  hypothesis  that  task-related  differences 
in  motor  time  constant  reflect  the  combined  effect  of  a  pilot 
bandwidth  limitation  plus  a  true  subjective  penalty  on  generating 
large  rates  of  change  of  control  force.  Specifically,  the 
control-rate  cost  coefficient  was  assumed  to  be  of  the  form 


G  +  G 
o  p 


( A- 1 ) 


where  G  is  the  cost  coefficient,  Gq  is  the  component  due  to  some 


minimum  motor  time  constant  Tm^ ,  and  Gp  is  the  component  related 


to  physical  activity  in  pounds  force  per  second.  The  base  level 


Tmo  may  be  interpreted  as  a  pilot  bandwidth  limitation,  or  as  the 


result  of  a  multiplicative  motor  noise  process  reflectina,  in 
part,  the  operator's  uncertainties  about  system  response  (Levison, 
1981;  Caglayan  and  Levison,  1980). 


The  first  test  of  this  hypothesis,  applied  to  data  obtained 
from  the  Gain/Bandwidth  Study  of  Levison  (1971),  yielded  encouraging 
results.  The  first  two  columns  of  Table  A-la  show  values  for 
motor  time  constant  and  corresponding  control-rate  cost  (G  of 
Equation  A-l)  for  the  simple  and  filtered  rate-control  tasks 
identified  as  Configurations  2,3,  and  4  in  Table  2-1.  Values 
shown  for  G  are  numerically  consistent  with  the  following  total 
performance  objective: 

J  =  a2  +  G  •  a2  ( A-  2 ) 

e  u 

where  J  is  the  total  "cost",  o  the  tracking  error  standard 

e 

deviation  score  in  degrees  visual  arc,  and  a*  the  control-rate 

-  u 

standard  deviation  score  in  pounds/second  of  control  force. 

Because  of  the  nonlinear  and  plant-dependent  relationship 

between  G  and  Tm  it  was  not  feasible  to  perform  a  simple 

regression  analysis  to  find  the  Tm  q  and  G^  that  would  provide  the 

best  joint  match  to  the  three  experimental  conditions.  Instead, 

the  value  Tmo=0.0704  found  for  the  simple  rate  dynamics  was 

assumed  to  reflect  the  intrinsic  bandwidth  limitation,  and  Gp 

was  then  computed  (separately  for  each  plant)  as  G  -  G(Tmo). 

Columns  4  and  5  of  Table  A-la  contain  values  for  G  and  G 

°  p 

identified  by  this  process.  Finally,  an  average  value  for  G 

P 

was  obtained  by  computing  the  geometric  mean  of  the  Gp  identified 

for  the  two  filtered  plants,  and  Equation  A-l  was  used  to  "predict" 

a  new  value  G  for  each  plant,  using  the  average  G^  and  the  plant- 

specific  G  . 

o 

Comparison  of  the  second  and  last  columns  of  Table  A-la 

A 

reveal  that  the  predicted  G  were  very  close  to  the  corresponding 
G  obtained  in  the  original  matching  exercise.  Recomputation  of 


Table  A- 1 .  Component  Analysis  of  the  Motor  Time  Constant 


a)  Simple  and  Filtered  Rate  Control 


Dynamics 


Tm  * 
o 


G  |  G 
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Simple  Rate  Control 

.0704 

7. 74E-3 

.  0704 

7. 74E-3 

0.0 

8.7  5E-3 

Rate  Control  + 

2  rad/sec  filter 

.  137 

1. 05E-3 

.0704 

1. 57E-5 

1. 03E-3 

1. 03E-3 

Rate  Control  + 

1  rad/sec  filter 

.174 

9. 92E-4 

.0704 

1. 95E-6 

9.90E-4 

1. 01E-3 

*By  assumption 
**Based  on  G^  =  1.01E-3 


b)  Rate,  Filtered-Rate ,  and  Acceleration  Control 


Simple  Rate  Control  I  .0704  7.74E-3  .0704  7.74E-3  I  0.0 


Rate  Control  + 

1  rad/sec  Filter 


Approximate 
Acceleration  Control 


*By  assumption 
**Based  on  Gp  =  4.81E-3 
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1. 27E-2 


1.88E-3I  2.44E-2  I  6.79E-3 


model-matching  errors  using  G  yielded  negligible  increase  over 
the  matching  errors  obtained  with  the  best-fit  G.  Thus,  the 
results  of  this  analysis  (based  on  data  obtained  from  a  single 
set  of  test  subjects)  support  the  hypothesis  that  changes  in  the 
identified  motor  time  constants  do  not  reflect  differences  in 
information-processing  capacity,  but  rather  a  combined  influence 
of  underlying  response  bandwidth  limitations  plus  a  consistent 
penalty  on  generating  control  activity. 

This  hypothesis  failed  for  support  when  the  analysis  described 
above  was  applied  to  the  following  set  of  conditions:  (1)  simple 
rate  control,  (2)  rate  plus  1  rad/sec  Butterworth  filter,  and  (3) 
approximate  acceleration  control  (Configuration  6  of  Table  2-1). 

The  various  cost  and  time-constant  parameters  identified  in  this 
analysis  are  shown  in  Table  A-lb.  Again,  the  motor  time  constant 
found  for  the  rate-control  task  was  taken  as  the  baseline  value 
for  Tm^,  and  non-zero  estimates  for  the  physical  cost  component 
G  were  derived  from  data  provided  by  the  remaining  two  tasks. 
Column  4  of  the  table  shows  that  the  two  estimates  so  obtained 
differed  by  more  than  an  order  of  magnitude. 

As  before,  the  geometric  mean  of  the  two  non-zero  estimates 
of  G  was  used  to  predict  new  cost  weightings  G  for  each  of  the 

ir 

three  plants.  Unlike  the  previous  analysis,  however,  these 
weightings  yielded  model-matching  errors  that  were  "significantly" 
greater  than  obtained  with  the  best-fitting  weightings.  Specif ical 
ly,  matching  error  ratios  (defined  in  Section  2.2.2)  were  1.3, 

5.0,  and  4.6,  respectively  for  the  rate,  f iltered-rate,  and 
acceleration  plants. 

Now,  there  is  no  guarantee  that  the  value  for  Tm  used  in 

o 

this  analysis  is  the  optimal  value;  there  may  be  an  alternate 
choice  that  provides  a  better  overall  match  to  the  collective  data. 


A  small  sensitivity  analysis  indicated,  however,  that  modifying 
Tm  would  improve  the  match  to  a  given  condition  only  at  the 
expense  of  degrading  the  match  to  at  least  one  other  condition. 
Therefore  it  seems  unlikely  that  fixed  values  can  be  found  for 
the  independent  parameters  of  Equation  A-l  that  will  satisfactorily 
account  for  the  apparent  task-related  changes  in  motor  time 
constant  revealed  in  Table  2-1. 

Although  the  acceleration-control  data  were  obtained  in  a 
different  experiment  (performed  by  a  different  experimenter)  than 
the  rate  and  filtered- rate  data,  there  was  considerable  commonality 
across  the  experiments: 

1.  Error  was  indicated  by  translation  of  an  electronically- 
presented  error  indicator  referenced  to  an  electronically- 
presented  zero  indicator. 

2.  Controls  were  hand-operated  and  nearly  isometric. 

3.  Subjects  were  well-trained ,  and  were  all  instructed  to 
minimize  mean-squared  error. 

While  some  differences  in  performance  capabilities  would  be 
expected  from  different  subject  populations,  an  order  of  magnitude 
difference  in  the  subjective  penalty  assigned  to  rate  of  change 
of  control  does  not  seem  reasonable  on  this  basis  alone.  Thus, 
we  tentatively  conclude  that  (1)  some  other  combination  of  band¬ 
width  and  physical  constraints  will  match  the  data  base,  or  that 
(2)  apparent  task-related  changes  in  the  motor  time  constant 
parameter  reflect  some  other  factor  limiting  perceptual-motor 
performance  that  is  not  directly  accounted  for  in  our  modeling 
philosophy. 


APPENDIX  B 

SUPPLEMENTAL  ANALYSIS  CONCERNING  EFFECTS  OF  PRACTICE 

Additional  experimental  and  model  results  relating  to  practice 
effects  are  presented  in  this  Appendix.  A  description  of  the  data 
base  and  a  summary  of  experimental  and  model  results  may  be  found 
in  Chapter  3. 

B.l  Effects  of  Practice  on  Frequency  Response 

Experimental  and  model  results  are  presented  as  follows  for 
individual  subjects: 

Figure  B-l  (5  parts):  static  group,  stable-plant  study. 

Figure  B- 2  (4  parts) :  motion  group,  stable-plant  study. 

Figure  B-3  (6  parts) :  unstable-plant  study 

Describing  function  data  are  plotted  only  where  signal/noise 
considerations  indicate  that  the  data  are  reliable  (Levison, 

1971).  Consequently,  some  of  the  curves  pertaining  to  early 
practice  conditions  span  a  relatively  restricted  frequency  range. 


Figure  B-l. 


Effects  of  Practice  on  Pilot  Frequency  Response 
Static  Group,  Stable  Plant  Study 
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B.2  Effects  of  Practice  on  Independent  Model  Parameters 

Tables  B-l  and  B-2  contain  model  parameters  identified  via 
the  unconstrained  and  constrained  search  procedures,  respectively, 
for  the  stable-plant  study.  Population  means  and  standard 
deviations  are  shown  along  with  data  for  individual  subjects. 

Parameters  for  subjects  participating  in  the  unstable-plant 
study  are  given  in  Table  B-3  for  the  unconstrained  search.  (The 
constrained  search  performed  on  the  unstable-plant  data  was 
performed  on  the  population  means  only,  not  on  data  obtained  from 
individual  subjects.)  The  rows  designated  as  "average”  contain 
the  parameters  identified  from  the  ensemble- averaged  data,  as 
contrasted  with  the  means  of  the  parameters  identified  for 
individual  subjects. 
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Table  B-l  .  Pilot  Parameters  Identified  from  the  Stable-Plant 
Data:  Unconstrained  Search 


Para] 

meter  | 

Subject 

Pe 

Pe  Pe 

Pm 

Td 

Tm 

a)  Static  Group,  Early 


CP 

-  5.6 

-18.7 

—  — 

-27.0 

.235 

DC 

-14.0 

-14.7 

— 

-49.5 

.169 

DS 

-10.0 

-14.0 

— 

-36.8 

.182 

TB 

-10.8 

-16.0 

— 

-55.6 

.198 

VS 

-11.9 

-14.5 

— 

-40.6 

.152 

MEAN 

-10.5 

-15.6 

— 

-41.9 

.187 

STD  DEV 

3.1 

1.9 

—  “ 

11.1 

.032 

.345 

.220 

.145 

.162 

.152 

.205 

.084 


b)  Static  Group,  Late 


CP  -21.6 

-16.3 

-29.4 

.161 

.169 

DC  -24.0 

-21.1 

— 

-41.0 

.280 

.127 

DS  -25.4 

-17.6 

— 

-35.9 

.206 

.131 

TB  -21.3 

-17.3 

— 

-56.5 

.218 

.112 

VS  -20.3 

-21.2 

— 

-40.3 

.238 

.138 

MEAN  -22.5 

-18.7 

— 

-40.6 

.221 

.135 

STD  DEV  2.1 

2.3 

10.1 

.044 

.021 

a)  Motion  Group,  Early 


CF 

-23.3 

-14.2 

-20.3 

-60.0 

.200 

1 

DM 

-16.4 

-16.5 

-18.6 

-60.0 

.200 

■ 

ML 

-  8.4 

-10.8 

-16.4 

-60.0 

.200 

HI  m 

RK 

-  5.3 

-14.9 

-22.6 

-60.0 

.200 

.109 

MEAN 

-13.4 

-14.1 

-19.5 

-60.0 

.200 

.121 

STD  DEV 

8.1 

2.4 

2.6 

.015 

b)  Motion  Group,  Late 


CF 

DM 

ML 

RK 

MEAN 
STD  DEV 


-30.7 

-18.4 

-25.4 

-60.0 

.200 

.0838 

-32.4 

-19.8 

-25.1 

-60.0 

.200 

.0770 

-32.8 

-  8.7 

-24.4 

-60.0 

.200 

.119 

-33.6 

-  7.8 

-24.7 

-60.0 

.200 

.118 

-32.4 

-13.  7 

-24.9 

-60.0 

.200 

.0995 

1.2 

6.3 

4.4 

—  — 

—  — 

.0222 

mm 

mu 

vVAV.v 


Subject 

Pe=P6 

Pm 

Td 

Tm 

a)  Static 

:  Group,  E 

arly 

CP 

-29.0 

.200 

DC 

-15.5 

-45.0 

.200 

DS 

-13.0 

-36.0 

.200 

TB 

-15.5 

-56.0 

.200 

.162 

VS 

-40.0 

.200 

.152 

MEAN 

-15.2 

-41.2 

.200 

.204 

STD  DEV 

1.4 

10.1 

—  — 

.083 

b)  Static  Group,  Late 


CP 

-19.0 

-29.0 

.200 

.170 

DC 

-18.5 

-45.0 

.200 

.125 

DS 

-36.0 

.200 

.130 

TB 

-17.5 

-56.0 

.200 

.112 

VS 

-19.5 

-40.0 

.200 

.138 

MEAN 

-18.9 

-41.2 

.200 

.135 

STD  DEV 

1.0 

10.1 

—  — 

.022 

c)  Motion  Group,  Early 

CF 

-18.5 

-60.0 

.200 

KfflH 

DM 

-17.5 

-60.0 

.200 

Kir 

ML 

-11.0 

-60.0 

.200 

RK 

-14.5 

-60.0 

.200 

.120 

MEAN 

-15.4 

-60.0 

.200 

.124 

STD  DEV 

3.4 

.015 

d)  Motion  Group,  Late 


CF 

-24.5 

-60.0 

.200 

.0836 

DM 

-24.5 

-60.0 

.200 

.0770 

ML 

-23.5 

-60.0 

.200 

.119 

RK 

-24.0 

-60.0 

.200 

.118 

-24.1 

-60.0 

.200 

.0994 

STD  DEV 

0.5 

.0222 

Table  B-3 .  Pilot  Parameters  Identified  from  the 
Unstable-Plant  Data 


Subject 

Pe 

P6 

a)  Early 

DMI 

-17.2 

-15.7 

KDH 

-15.0 

-21.  3 

KJW 

-20.6 

-21.9 

LEK 

-16.3 

-18.  8 

SAH 

-17.9 

-21.0 

TMV 

-20.7 

-21.6 

MEAN 

-18.0 

-20.0 

STD  DEV 

2.3 

2.4 

AVERAGE 

-16.9 

-19.9 

b)  Late 

DMI 

-23.4 

-22.1 

KDH 

-25.1 

-21.7 

KJW 

-26.3 

-23.2 

LEK 

-24.7 

-21.2 

SAH 

-23.7 

-22.4 

TMV 

-22.5 

-22.7 

MEAN 

-24.3 

-22.2 

STD  DEV 

1.4 

0.7 

AVERAGE 

-23.8 

-22.0 

Parameter  . 
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-45.3 
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B. 3  Effects  of  Search  Constraint  on  Matching  Error 

Model  parameters  were  initially  identified  without  constraints 
(other  than  the  constraint  that  all  parameters  be  numerically 
positive) .  Data  from  the  stable-plant  study  were  subsequently 
re-analyzed  subject  to  two  levels  of  constraint.  For  constraint 
conditions  Cl,  time  delay  was  fixed  at  0.2  seconds  for  all  subjects. 
The  motor  noise/signal  ratio  was  fixed  for  each  static-group  subject 
at  the  average  of  the  early  and  late  values  previously  identified; 
whereas  the  motor  noise/signal  ratio  was  fixed  at  -60  dB  for  all 
motion-group  subjects.  Constraint  level  C2  included  the  constraints 
of  Cl,  plus  the  additional  constraint  that  a  single  observation 
noise/signal  ratio  be  applied  to  all  observational  variables 
(i.e.,  Pe  =  Pe  for  the  static  group,  Pe=Pe=Pe  for  the  motion 
group) . * 

Table  B-4  shows  matching  errors  along  with  matching  error 
ratios  relative  to  the  unconstrained  error,  for  the  subjects 
participating  in  the  stable-plant  study.  Matching  error  ratios 
are  absent  for  one  case  —  Subject  ML,  motion  group,  early 
practice  —  because  of  a  failure  for  the  OCM  algorithms  to 
converge  during  the  unconstrained  search.  All  calculations  were 
performed  on  three-place  numbers  and  then  rounded  off  to  two  digits 
for  presentation. 

A  certain  amount  of  caution  should  be  exercised  when  inter¬ 
preting  the  matching  errors  shown  in  Table  B-4,  as  these  errors 
are  determined  by  the  variability  in  the  data  as  well  as  by  the 
actual  degree  of  mismatch  (see  Section  2.2.2).  Nevertheless, 
the  larger  errors  shown  for  the  early-practice  condition  confirm 

Because  the  quasi-Newton  search  routine  implemented  at  the  time 
of  this  analysis  did  not  specifically  allow  the  user  to  constrain 
all  noise/signal  ratios  to  a  single  value,  the  C2  search  (a  2- 
parameter  search)  was  performed  manually.  Hence,  values  for  noise/ 
signal  ratio  shown  in  Table  B-2  are  quantized  to  the  nearest  0.5  dB 
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Table  B-4.  Effects  of  Search  Constraints  on  Matching  Error 


Early 

Late 

Matching  Error 

Matching  Error 

Subject 

Constraint* 

Actual  |  Ratio 

Actual  j  Ratio 

a)  Static  Group,  Stable-Plant  Study 


CP 

U 

Cl 

C2 

38 

40 

42 

1.0 

1.1 

7.4 

7.  8 
9.1 

1. 1 

1.2 

U 

18 

—  — 

9.1 

— 

DC 

Cl 

18 

1.0 

10 

1.1 

C2 

19 

1.1 

12 

1.  3 

U 

1.8 

mam 

8.1 

— 

DS 

Cl 

1.9 

8.1 

1.0 

C2 

3.6 

i  ■ 

12. 

1.5 

U 

19 

— 

2.6 

— 

TB 

Cl 

20 

BIliK 

2.6 

1.0 

C2 

21 

1.1 

_ 

3.  8 

1.5 

U 

13 

mam 

mgm 

— 

VS 

Cl 

14 

Bh 

1.0 

C2 

16 

mSM 

1.1 

b)  Motion  Group,  Stable-Plant 

Study 

u 

17 

— 

6.2 

—  — 

CF 

Cl 

17 

1.0 

6.2 

1.0 

C2 

18 

1.1 

7.0 

1.1 

U 

7.5 

—  — 

6.6 

—  — 

DM 

Cl 

7.  5 

1.0 

6.6 

1.0 

C2 

8.5 

1.1 

8.5 

1.3 

U 

—  — 

— 

11 

— 

ML 

Cl 

14 

— 

11 

1.1 

C2 

17 

— 

14 

1.3 

u 

6.2 

— — 

5.5 

—  _ 

RK 

Cl 

12 

2.0 

6.2 

1.1 

C2 

33 

5.3 

8.2 

1.  5 

♦Constraints  were  as  follows: 

U:  unconstrained 

Cl:  Time  delay  fixed  at  0.2  seconds,  all  runs.  Motor  noise/ 

signal  ratio  fixed  at  average  of  early  and  late  for  static 
group,  -60  dB  for  motion  group. 

C2:  Same  as  Cl,  plus  Pe=P6  for  static  group,  Pe=P6=Pe  for 
motion  group. 
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the  subjective  impressions  obtained  from  visual  inspection  of 
Figures  B-l  and  B-2;  namely,  that  the  OCM  tended  to  provide  a 
better  match  to  the  data  obtained  late  in  the  training  phase. 

Table  B-4  shows  that  in  16  out  of  17  cases,  the  Cl  constraint 
(fixed  delay  and  motor  noise/signal  ratio)  increased  the  matching 
error  ratio  by  less  than  15%.  A  two- fold  increase  was  found  for 
the  remaining  case  (RK,  motion  group,  early) ,  but  the  relative 
lack  of  valid  describing  function  data  (see  Figure  B-2d)  for  this 
condition  renders  the  model-match  suspect.  In  summary,  fixing  time 
delay  and  motor  noise/signal  ratio  had  no  significant  influence 
on  model-matching  error,  and  model  analysis  made  with  this  constraint 
was  justified. 

Constraining  observation  noise/signal  ratios  to  be  uniform 
across  perceptual  variables  tended  to  degrade  the  matching  error 
by  a  greater  amount,  but  generally  not  enough  to  be  "significant" 
according  to  the  error  ratio  criterion  of  2.0.  Table  B-4  shows 
that  this  constraint  yielded  a  matching  error  ratio  of  under  1.25 
for  about  half  the  cases,  over  2.0  in  only  two  cases. 

T-tests  were  performed  on  paired  differences  between  error 
and  error-rate  noise/signal  ratios  for  the  stable-plant  (static 
group  only)  and  performance-analyzer  subject  populations.  A  weak 
significance  level  (alpha=.05)  was  found  for  the  performance- 
analyzer  subjects  late  in  training;  no  significance  (alpha  less 
than  0.05)  was  found  for  the  remaining  three  tests. 

The  results  of  the  significance  tests,  therefore,  generally 
justify  the  uniform- ratio  constraint  applied  to  this  data  base. 
Nevertheless,  the  noise  ratio  differences  did  show  consistent  trends. 
With  few  exceptions,  the  noise/signal  ratios  associated  with 
perception  of  error  were  greater  than  error-rate  noise/signal 
ratios  early  in  training,  whereas  the  reverse  trend  was  found  in 
the  data  obtained  late  in  training. 
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